Microsoft Azure IaaS Full BizTalk Domain Setup PowerShell Scripts

Microsoft Azure IaaS Full BizTalk Domain Setup PowerShell Scripts

Download this sample using the download button below.

Download Now

Change Log

1.0 (06/02/2014) – Initial release.
1.1 (06/10/2014) – Better VNet support, added per VM size configuration, auto log off when configuration is complete, added note that base VM name must be all lower case.

 

Important Points to Note

1. These scripts use Premium IaaS Images.  If you are not using a subscription tied to a MSDN account you will occur significant changes.

2. Due to legal issues related to the Microsoft provided BizTalk Provisioning Tool, I am unable to allow people to download it.  The BizTalk 2013 Images on IaaS contain an older, none working version.  Because of this some additional manual steps are required to complete the auto BizTalk configuration.  I have made the auto configuration an optional step that is enabled by a Flag in the variables file.  The default is false.  I suggest everyone spend 20 minutes to get the files from a developer image using the instructions below until we have an updated IaaS image that includes the files or until we are able to download them from Microsoft.  See below information on “To enable full end to end auto configuration do the following (Note – must be a MSDN users)”

3. The $baseVMName variable must be 8 characters or less and all lower case.

4. If the scripts fails, ensure you run 99_RemoteAll before re-running the scripts. 

5.  Due to limitations with PowerShell’s support for Virtual Networks, the 99_RemoteAll script will remove all Virtual Networks that do not have resources attached to them.  Ensure this is what you want before you run the script.  If you have existing Virtual Networks in use, the 99_RemoveAll script will fail on the removal step and the Virtual Network will need to be removed manually.


This is what the scripts do:

  • Creates a Virtual Network, Affinity Group, and Azure Storage account
  • Create a Domain Controller with the BizTalk Groups and Users created
  • Creates a SQL Server and assigns the correct permissions for the BizTalk users
  • Creates two BizTalk Servers
  • Optional – Downloads a bunch of helper files to configure the BizTalk Group
  • Optional – Installs Classic Shell (yes, I can not live without a Start button)
  • Optional – Installs BizTalk 2013 CU2
  • Downloads Remote Desktop connections to the newly created VMs

 

To run the scripts:

  • Review the prerequisites below.
  • Open variables.ps1 and review the variables.  Only the top 8 should need to be looked at or updated.  Of these, only 2 values must be changed.
    • $baseVMName = ‘<abcd1234>’                                 # UPDATE – Must be Globally Unique and 8 characters or less.  All lower case.
    • $subscriptionName = ‘<subscription name>’              # UPDATE – This is the name of your subscription from the publishing file
    • $setupDir = “C:\BizTalkGurus”                                   # OPTIONAL UPDATE – Used for files on the remote and local servers
    • $basePathToScripts = ‘<path to scripts>’                    # OPTIONAL UPDATE – This is used if you run the scripts through an IDE
    • $workStorgeAccount = ‘<update for auto configure>’    # OPTIONAL UPDATE – Update with the storage location of the helper files for auto configuration
    • $installCU2 = ‘false’                          #OPTIONAL UPDATE – Must upload BizTalk CU2 exe to the root of the Storage Account above
    • $installClassicShell = ‘false’              #OPTIONAL UPDATE – Must upload classic shell version 4.1 exe to the root of the Storage Account above
    • $installBizTalkProv = ‘false’               #OPTIONAL UPDATE – Must upload 5 files from BizTalk 2013 Dev box to the root of the Storage Account above
  • Run Master.ps1 in an elevated PowerShell window.  If running in an IDE it is best to run the script from the command windows rather than hitting Run. 

The end result will look something like this:


What is included

Prerequisites

  • Tested with PowerShell late- May release.  PowerShell versions before March (sorry can’t keep the versions straight) will fail in the VM creation because static IP Addresses are assigned to the VM’s in the network. 
  • Ensure you have a working PowerShell installation before running.  This can be tested by running Get-AzureLocation to ensure it returns values.  Details on how to set this up can be found here.
  • Scripts must be ran elevated. 

Main PowerShell Scripts

  • Master.ps1 – This is the only file you need to run.  Ensure you run from an elevated PowerShell window.  Takes a bit over an hour for full configuration.
  • variables.ps1  – This is the only file you need to update.  You should only need to look at the first 8 parameters. 
  • 00_Create_Network.ps1 – This script creates the affinity group, storage account, and virtual network.
  • 01_Create_PDC.ps1 – This script create the domain controller by promoting a windows server using remote PowerShell. It also creates all the groups and uses needed for the BizTalk configuration.
  • 02_Create_Servers.ps1 – This script creates the SQL and BizTalk Servers using the Enterprise Edition of the Microsoft provided images. 
  • 03_Configure_Servers_RemoteProp.ps1 – This script uses remote PowerShell to complete additional configuration on the SQL and BizTalk Servers.  If flags are set, files will be downloaded to the servers to complete configuration.
  • 04_Configure_Servers_LaunchRDP.ps1 – This script will launch the Remote Desktop sessions needed to complete the auto configuration.
  • 05_RemoteDesktop – Downloads the remote desktop connect for each server.  This is stored inside a new folder called RemoteDesktop in the scripts folder.
  • 99_RemoveVM.ps1 – End-to-end clean up script.  Removes all items created based on the variables file.  Note – due to limitations in Virtual Networks only run this if you have no other virtual networks on your subscriptions.   If you do, you should comment out the virtual network section of this script.
  • SaveMoney_StopAll.ps1 – Scripts to stop and deallocate all the Azure Virtual Machines based on the values in your variables file.
  • SpendMoney_StartAll.ps1 – Scripts to start all the Azure Virtual Machines based on the values in your variables file.
  • Configs folder – NetworkConfig.xml – This is the XML Configuration used to create the virtual network in Azure.
  • Helper Files – Helper files are only needed if you plan to auto-configure BizTalk, auto install CU2, or auto install Classic Shell

    • multinodeconfigDemo_DomainHosts.xml – XML file used by the BizTalk Provisioning tool to auto configure the BizTalk group.  No changes are needed to this file.
    • rdp.exe – Use to auto launch a Remote Desktop session to complete the BizTalk configuration. 
    • RunBizTalkTask_Domain.xml – File used to create a Windows Scheduled Task.
    • RunLocalClient.ps1 – PowerShell script to be ran locally on the First BizTalk Server in the group. 
    • RunLocalClientServer2.ps1 – PowerShell script to be ran locally on all Other BizTalk Server in the group. 
    • StartPowershell.exe – Helper exe to enable Windows Scheduled Task to easily run a PowerShell script in elevated mode.
    • Optional – ClassicShellSetup_4_1.exe – You need to be download from http://www.classicshell.net/ and uploaded to your container.   Ensure the version names match what is in the 03_Configure_Servers_RemoteProp.ps1 file.
    • Optional – BizTalk 2013 CU2 – This needs to be downloaded yourself and uploaded to your own public Azure container.
    • Optional – BizTalk.Provisioning.files– This needs to do downloaded yourself and uploaded to your own public Azure container.


To enable full end to end auto configuration do the following (Note – must be a MSDN users):

  • Put all helper files on your own public Azure storage account container
  • Update $workStorgeAccount in the variables file to the full address of your storage container
  • To enable auto install BizTalk 2013 CU2

    • Download BizTalk 2013 CU2 exe and upload it the root of your storage account
    • Update $installCU2 in the variables file to “true”
  • To enable auto BizTalk configuration

    • Create a BizTalk 2013 Development Virtual Machine in Microsoft Azure (script here)
    • Copy the 5 files in the C:\BizTalk_Provisiong folder
    • Upload the 5 files to the root of your storage account


If you do the auto configuration (witch is highly recommended for anyone who plans to run the scripts more than once) your end result will look like this.

 

Known Issues and Limitations:

  • At present the auto configuration is only possible if you download and host the BizTalk Provisioning files yourself.  This will change in the future and I will release a new set of scripts once the process is cleaner.
  • Every once in a while Azure is really slow.  I have added some speed breaks in the scripts to better account for this.
  • Every once in a while the auto configuration fails.  The servers get joined to the group by no hosts are created.  At that point, they should just be created manually or remove and restart all the scripts from scratch.
  • If you have an existing virtual network associated with your subscription the 99_RemoveVM script will delete it or fail trying.  This is a limitation in PowerShell. 

 

Enjoy and I look forward to community feedback!

%ufeff
BizTalk 2013 Development Azure VM Creation Script Download

BizTalk 2013 Development Azure VM Creation Script Download

Download this sample using the download button below.

Download Now

Important points to note:

– This will only work for an Azure subscription linked to an MSDN account.

– You are charged money (against your MSDN allowance) when the VM is running and stopped.  It needs to be Stopped – Deallocated in order to stop the billing.  To do this, use the portal to shut down the VM or use StopVM PowerShell script to turn it off. 

The scripts does the following:

  1. Creates a VM in Azure using the datacenter and size you specify.
  2. Auto configures your BizTalk development server to create a Receive, Send, Processing, and Tracking host.
  3. Downloads the remote desktop connection for you so all you need to do is click.
  4. Includes RemoveVM, StopVM, and StartVM scripts for easy admin.

To run the script only 2 values need to be updated.  In all four scripts (MakeVM, RemoveVM, StopVM, StartVM) update the following at the top of the file:

$baseVMName =  ‘any123’                               # UPDATE 1 – this needs to be globally unique, like initials and 4 random numbers
$subscriptionName =  ‘MSDN Subscription’     # UPDATE 2 – this is your subscription name

Make sure you review the other variables as well and make changes as needed. 

I’m currently hosting some supporting files on a public azure file store.  At some point I might take that down.  If so, you will need to upload the files to your own queue and update the MakeVM script.  I have included the 5 helper files in the Helper folder.

Windows Azure REST API Simple Sample and Tool

Windows Azure REST API Simple Sample and Tool

Download this sample using the download button below.

Download Now

This tool is intended to get you moving in the right direction and not intended to solve all your problems out of the box. You will need to use the Windows Azure REST API Reference guide when working with this tool.

I have provided a few sample Request Body templates for Creating a Hosted Service, Creating an Azure Storage Account, Adding a Virtual Machine Disk, and Creating a Virtual Machine (note to create a VM is a multi-step process, see below). Others can easily be added by using the online reference guide.

To use this sample tool you need to configure some basic information inside the App.Config file. You need to set the path to your management certificate, your subscription id, and the path to the folder location of your POST bodies.

    <!--  Enter the full path to the Windows Azure Management Certificate. More details at http://msdn.microsoft.com/en-us/library/windowsazure/ee460782.aspx -->
    <add key='CertificatePath' value='C:\DemoFolder\yourcert.cer'/>

    <!-- Subscription ID for the account and must match Management Certificate -->
    <add key='Subscription' value='your account id' />

    <!--  Default folder location of REST API Post Bodies.  This exists to save time selecting the Post Body file. -->
    <add key='PostBodiesFolder' value='C:\DemoFolder\WindowsAzureRESTApiHelper\WindowsAzureRESTApiHelper\RESTAPIBodies\'/>


How to Use this Sample Tool

1. Once the App.config is setup simply launch the tool. It was built with Visual Studios 2010 but should upgrade to 2012 without issues.

2. Select one of the GET, POST, or DELETE radio buttons.

3. Select your basic action from the drop down or paste in your URL into the text box.

 

4. Click on “Make A REST Service Call” to complete your request.

 

5. If you are doing a POST, you will be prompted to select the body of your request.

 

6. If you do a POST or a DELTE that is an asynchronous process a Request ID is returned in the Response Header. The tool will detect this and auto populate a URL to check the result of the request.

 

7. You can keep clicking the “Make REST Service Call” button to check the results of the submitted operation until it is competed.

 

It is that simple!!! Like I said before, this tool is intended to be a starting point for someone new to working with the Windows Azure REST API’s.


Other useful pieces of information

1. To create a new Virtual Machine you need to do the following: First, create the Service using Create Hosted Service. Second, you can use the Quick Create using that Service to create the Virtual Machine.

2. If you plan to work with Virtual Machines using the API, read this blog post about the URLs.

3. If you run into issues the best way to confirm the URL and Post Body is to perform the actions using PowerShell with Fiddler running. This will show the URL and exact Post Body.

Windows Azure Virtual Hard Disk (blob storage) Cross Account Copy, Lease Break, and Management Tool

Windows Azure Virtual Hard Disk (blob storage) Cross Account Copy, Lease Break, and Management Tool

Download this sample using the download button below.

Download Now

Basic Steps for Cross Account VHD Copy:

  1. Shut down your Virtual Machine.
  2. Delete your Virtual Machine.
  3. Delete the associated Image or Disk inside the Virtual Machine Portal. This does not remove the underlying blob of the vhd.
  4. Copy the blog from one storage account to another.
  5. Add your vhd back and recreate the virtual machine.

Known Issues (per my research) with VHD Copying– make sure you understand these issues and limitations. I assume most are related to preview issues.

  • If you copy a vhd that is Leased (configured as an Image or Disk inside the Virtual Machine Portal) the Source Disk will become unusable.
  • If you copy a vhd that is Leased (configured as an Image or Disk inside the Virtual Machine Portal) the Destination Disk will likely not work. I would say about 75% of the time they do not work. I just re-tested this tonight and they worked.
  • Copying from one data center to another seems to be as slow as downloading the file.
  • When downloading the file, make sure you do not close the form or the download will stop.
  • Copy Status does not seem to be working. I either get NULL or Pending all the time. Probably a user error on my part…. The best way to find out if the copy is complete is to check the file size on the destination. It is 0 MB until the copy is complete.

Features:

  • Visual view of vhds contained inside a Storage Account
  • Simple, one click Delete of vhds
  • Simple, one click Lease Break of vhds (for orphaned vhds – see more here)
  • One click download of a vhd, tested to work with Hyper-V (downloads at a rate of about 3 MB/sec)
  • Support for easily making a copy of a VHD within the same Azure account
  • Support for making a vhd copy across Window Azure Accounts within the same region within a few seconds
  • Support for making a vhd copy to any account in any region (slow when out of region)
  • Prevents you from making a copy of a vhd with an existing Lease
  • Uses the Windows Azure Storage API version 1.7.1 (github)

Advanced Features (with REST API Management Certificate):

  • Visual view of all vhds contained inside a single Windows Azure Account
  • See a “***” by vhds inside the Source storage account
  • List all custom user images inside a single Windows Azure Account

Missing Features (i.e. items I hope to add later):

  • Ability to upload a vhd
  • Ability to get details on an Image

 

Quick How-To’s:

How to work with just one Storage Account:

Set just a Source Configuration inside the app.config file. I also have a check box for “source only” that will try to auto determine what you want to do. The default is Source Only.

How to Copy a vhd into the same account:

Inside the app.config file for the tool, set the Source and Destination to the same Storage Account.

How to Copy a vhd into another account in the same region (inter-account or cross-account):

Inside the app.config file for the tool, set the Source to the Storage Account that has the source vhd and set Destination to Storage Account that you want for the Destination. The destination container vhds does not need to exist. The tool will create it for you. Click “Copy”. The Destination list should refresh and the copy already be compete. You will know it is complete when the size of the vhd is not 0.

How to Copy a vhd cross region:

Inside the app.config file for the tool, set the Source to the Storage Account that has the source VHD and set Destination to Storage Account that you want for the Destination. The destination container vhds does not need to exist. The tool will create it for you. Select a vhd in the List box that is not lease. Change the vhd name if you wish. Click “Copy”. Click on “List All VHDS” on the top right until you see the file size of the Destination change from 0 to the size of the VHD. This will take a long time, up to 8 hours. I would not close the form until I have confirmed it was completed.

How to Download a Source vhd:

Inside the app.config file for the tool, set the Source to the Storage Account that has the source VHD. Click on “List All VHDS”. Select the VHD you want to download inside the List Box. Click on “Download”. You will be prompted for the download location. Make sure you have enough disk space. Leave the tool open until you see a Completed notice on the top right of the tool. This will take up to 8 hours. Do not close the form or the download will stop.

How to Break a Lease or Delete a vhd on the Source or Destination:

Inside the app.config file for the tool, set the Source and Destination to the Storage Accounts that has the VHDS. Click on “List All VHDS”. Using the List Box, select a single vhd from the list. Click on “Break VHD Lease” to break the existing lease or “Delete VHD” to delete the vhd. Note: this operations are not undoable. Using these incorrectly can result in loss of data or vhds.

Setting Up Continuous Integration, Unit Tests, and MSI Creation in BizTalk 2009 Sample Code

Setting Up Continuous Integration, Unit Tests, and MSI Creation in BizTalk 2009 Sample Code

Download this sample using the download button below.

This sample code outlines how to set up Continuous Integration, Automated Unit Tests, and to create an MSI Package using Team Foundation Server 2008 (TFS) with BizTalk Server 2009.

At a high level, this is what is happening in the end to end process.

A file is updated and checked in -> A build is started -> The build completes -> Defined unit tests are ran -> a MSI Package is created

This sample code goes along with a step-by-step blog post that outlines this process.

That blog post can be found here: http://www.biztalkgurus.com/blogs/biztalk-integration/2009/04/16/setting-up-continuous-integration-automated-unit-tests-and-msi-packaging-in-biztalk-2009/

 

Host Instance Creation and Adapter Configuration Tool for BizTalk

This tool will help you design and build your BizTalk solutions by making it easier to consistently separate out your processes into multiple hosts even on your developer boxes.  It will also make sure all your developers and environments are using Hosts with the same names.  This can be a total nightmare to manage and maintain.

This tool is fed by a simple config file to set up the Hosts, install Host Instances, and assign Adapters to these new Hosts.  This tool has the ability to prompt for a user password and reset all the Adapters back to the default Host (this was used extensively when I was testing the tool).

This tool should work with BizTalk Server 2004, BizTalk Server 2006, and BizTalk Server 2006 R2.  I have not tested it running against any remote servers. 

For more information, see the full blog post at: http://www.biztalkgurus.com/biztalk_server/biztalk_blogs/b/biztalk/archive/2008/04/30/biztalk-host-creation-and-adapter-assignment-tool-available-for-download.aspx

BizTalk Advanced Orchestration Concepts and Best Practices Slides and Demos

These are the slides and demos from my Session at the SOA and BPM Conference in Redmond in October, 2007.   

The demos show Dynamic Mapping, Self-Correlation Ports, Untyped Messages, and Convoys.  These demos are “as is” and may need some configuration changes to work on your system.

PowerPoint 2007 is required to view the slides.

Session Abstract:
Business processes are a required component in most Enterprise Integration solutions today. Business processes are modeled, designed, and built inside BizTalk Server using Orchestrations. Orchestration can range from a few simple shapes to a complex multi Orchestration, Transactional process. This session focuses on highlighting Advanced Orchestration features and best practices that can be used to shorten development time and increase overall Business Process reusability. Topics covers are: Untyped Messages, Dynamic Transforms, Starting Orchestration, Passing Port Parameters, and Convoys.”

Advanced Microsoft BizTalk 2006 R2 Concepts Slides and Demos

These are the slides and demos from my Tech Ed Session from Tech Ed 2007 in Orlando.   

The demos show Dynamic Mapping, Self-Correlation Ports, Untyped Messages, and WCF Calls in Messaging.  The WCF demos will require BizTalk Server R2.  These demos are “as is” and may need some configuration changes to work on your system.

PowerPoint 2007 is required to view the slides.

Session Abstract:

“Business processes are a required component in most Enterprise Integration solutions today. Business processes are modeled, designed, and built inside BizTalk Server 2006 R2 using Orchestrations. Orchestration can range from a few simple shapes to a complex multi Orchestration, Transactional process. This session focuses on highlighting key Orchestration features to shorten development time and increase overall Business Process reusability. Also, messaging-only scenarios using WCF are discussed, and the power of BizTalk as a Web service routing system is shown. Topics covers are: Untyped Messages, Dynamic Transforms, Starting Orchestration and Passing Port Parameters, using Helper .NET Components, and Message Only WCF Calls.

First Look at the WCF Adapter WCF Sample Code

This sample shows a basic WCF Service using Basic HTTP Binding. This service is hosted inside IIS and uses a WCF Receive Location to process WCF messages inside BizTalk.

This code was built using a beta version of BizTalk 206 R2.  The concepts are the same in the RTM release of BizTalk 2006 R2 but this code may not work.

Watch the video related to this sample here: http://www.biztalkgurus.com/biztalk_server/biztalk_2006_r2/m/biztalk_2006_r2_samples/32391.aspx

First Look at the WCF Adapter in BizTalk 2006 R2 Video

Note: This video is based on BizTalk Server 2006 R2 BETA

This nearly 13 minute video gives a quick, high level First Look at BizTalk Server 2006 R2 WCF Adapter using the Basic HTTP Binding. This is intended to show how easy it is to use BizTalk 2006 R2 and WCF to build applications.

Note that this is based on Beta code and is subject to change in future releases.

This content should be current for the RTM release of BizTalk 2006 R2 and beyond.