Build a Multi-Server, Configured, BizTalk 2013 R2 Domain in Windows Azure IaaS in less than 1 hour!

Build a Multi-Server, Configured, BizTalk 2013 R2 Domain in Windows Azure IaaS in less than 1 hour!

A little over 2 years ago I released a set of PowerShell scripts for auto creating a full Multi-Server BizTalk 2013 Domain inside Azure IaaS.  I spent 100’s of hours on these scripts to get them working correctly.  You can view my session at TechEd 2013 US online for more details.

I have made updates to these scripts and updated them to support BizTalk 2013 R2 and SQL 2014.  These scripts will automatically created a clean isolated 2 BizTalk Server 2013 R2 Domain using SQL 2014.  All domain user accounts are auto created.  BizTalk group and host settings are created.  The only manual intervention that is needed is to log into each server and run a PowerShell script.  In less than an hour you can have a fully configured BizTalk 2013 R2 Domain!  And since it is scripted, you can create a clean environment from scratch anytime you need one.

Note: Running these scripts will created Virtual Machines inside Windows Azure.  You will be charged for the Virtual Machines.  Make sure you understand the costs before running these scripts.

Why not use an Azure Resource Manger Template for this?  Great question!  I tried.  I spent about 40 hours trying to port these scripts into an Azure Resource Manger Template with no success.  I was unable to get the BizTalk and SQL Servers to join the domain.  Azure Resource Manger PowerShell has a new set of commands that I could of used, but they are in the process of changing – so for now keeping the scripts as-is seemed the best course of action.

You can review and download the updated scripts on MSDN – Automatic Multi-Server BizTalk 2013 R2 Domain Creation PowerShell Scripts for Azure IaaS.  If you like the scripts please rate them with a 5-star.

To run the scripts inside Visual Studio 2013 / 2015 do the following:

  1. Download them from MSDN – they are inside a Visual Studio 2013 Solution
  2. Open Visual Studio as an Administrator and open the Solution
  3. Update 3 values inside variables.ps1  It is the top 3 variables in the file.
  4. Update the top variable inside MAIN_MASTER.ps1 It is the first variable in the script.
  5. Right-click on MAIN_MASTER.ps1 and select "Execute As Script".  Wait about 45 minutes. 
  6. Log into BizTalk 02, open C:\BizTalk_Provisioning\ReadMe.docx.  Follow the instructions.  Do not close the Remote Desktop session.  The PowerShell window will open and close right away.  See details below.
  7. Log into BizTalk 01, open C:\BizTalk_Provisioning\ReadMe.docx.  Follow the instructions.  Wait about 10 minutes. Note that you have to run the last command twice.  See details below.

To run the scripts through PowerShell IDE dot he following:

  1. Download them from MSDN – they are inside a Visual Studio 2013 Solution
  2. Open Main_Master.ps1 and variables.ps1 inside PowerShell IDE, make sure to open PowerShell IDE as an Administrator
  3. Update 3 values inside variables.ps1
  4. Update the top variable inside MAIN_MASTER.ps1
  5. Run MAIN_Master.ps1 as an Administrator
  6. Once complete, Log into BizTalk 02, open C:\BizTalk_Provisioning\ReadMe.docx.  Follow the instructions.  Do not close the Remote Desktop session.  The PowerShell window will open and close right away.   See details below.
  7. Log into BizTalk 01, open C:\BizTalk_Provisioning\ReadMe.docx.  Follow the instructions.  Wait about 10 minutes. Note that you have to run the last command twice.  See details below.

This download contains the following scripts:

  • MAIN_Master.ps1 – This is the main scripts that calls all the other scripts that do all the work.
  • MAIN_RemoveVM.ps1 – This script will remove and delete everything created inside the domain.
  • MAIN_SaveMoney_StopAll.ps1 – This script to turn off all the VM’s in the domain.  You need to update 1 value in this file to run it.
  • MAIN_SpendMoney_StartAll.ps1 – This script to turn on all the VM’s in the domain.  You need to update 1 value in this file to run it.
  • variables.ps1 – Core variable file, you need to update 3 values in here.
  • Various other helper scripts and files

Detailed Post Script Configuration Steps (this is outlined in the ReadMe.docx downloaded to each server)

On BizTalk 02:

  1. Once the PowerShell scripts are complete log into the BizTalk 02 server with the domain admin account.
  2. Open a PowerShell windows as Administrator.

  3. Change directories as follows “cd c:\BizTalk_Provisioning”
  4. Run the following command “MicrosoftCloudProvisioningLocalService.exe” – you should see this running as a service.

  5. Leave the Remote Desktop Session open and go to the BizTalk 01 Server.

On BizTalk 01:

  1. Once the PowerShell scripts are complete and the service is started on BizTalk 02, log into the BizTalk 01 server with the domain admin account.
  2. Open two PowerShell windows as Administrator.

  3. In the first window, change directories as follows “cd c:\BizTalk_Provisioning”
  4. Run the following command “MicrosoftCloudProvisioningLocalService.exe” – you should see this running as a service.

  5. In the second window, change directories as follows “cd c:\BizTalk_Provisioning”
  6. Run the following command “.\Microsoft.Cloud.Provisioning.Client.exe .\multinodeconfigDemo.xml” – you should see several pop-up windows.  This should fail.

  7. Run the following command again “.\Microsoft.Cloud.Provisioning.Client.exe .\multinodeconfigDemo.xml” – you should see several pop-up windows again but this time it should Complete.

  8. The end result is a fully configured, multi-server BizTalk Domain inside Windows Azure!

Make sure you use the provided scripts to start, stop, and remove the artifacts created.  I look forward to your feedback!

End to End BizTalk Domain Setup in Windows Azure IaaS Scripts

Got a little over an hour?  Get a full featured multi-server BizTalk 2013 domain running in Windows Azure IaaS!  It is fully configured with Send, Receive, Processing, and Tracking hosts all ready to go.  Since it is scripted, it is repeatable so you can tear it down when you are done and create a brand new fresh environment the next time! 

Cost: For a MSDN user the cost is around $0.55 per hour for the compute time with Medium VMs!  Monthly charges may be incurred for storage, bandwidth, and transaction but in my experience these have be next to nothing.  The costs for non-MSDN users will be significantly higher as these are premium images in Azure.

As promised at the BizTalk Summit in London in March, here are the Windows Azure IaaS Full BizTalk Domain Setup scripts.  Of course these scripts can be used to create any type of domain, not just for BizTalk. And better yet, you only need to update as little as 2 variables file to be able to run them!

Download scripts: Microsoft Azure IaaS BizTalk Domain Setup PowerShell Scripts

Important Points to Note

1. These scripts use Premium IaaS Images.  If you are not using a subscription tied to a MSDN account you will occur significant changes.

2. Due to legal issues related to the Microsoft provided BizTalk Provisioning Tool, I am unable to allow people to download it.  The BizTalk 2013 Images on IaaS contain an older, none working version.  Because of this some additional manual steps are required to complete the auto BizTalk configuration.  I have made the auto configuration an optional step that is enabled by a Flag in the variables file.  The default is false.  I suggest everyone spend 20 minutes to get the files from a developer image using the instructions below until we have an updated IaaS image that includes the files or until we are able to download them from Microsoft.  See below information on “To enable full end to end auto configuration do the following (Note – must be a MSDN users)”

This is what the scripts do:

  • Creates a Virtual Network, Affinity Group, and Azure Storage account
  • Create a Domain Controller with the BizTalk Groups and Users created
  • Creates a SQL Server and assigns the correct permissions for the BizTalk users
  • Creates two BizTalk Servers
  • Optional – Downloads a bunch of helper files to configure the BizTalk Group
  • Optional – Installs Classic Shell (yes, I can not live without a Start button)
  • Optional – Installs BizTalk 2013 CU2
  • Downloads Remote Desktop connections to the newly created VMs

To run the scripts:

  • Review the prerequisites below.
  • Open variables.ps1 and review the variables.  Only the top 8 should need to be looked at or updated.  Of these, only 2 values must be changed.
    • $baseVMName = ‘<abcd1234>’                                 # UPDATE – Must be Globally Unique and 8 characters or less.
    • $subscriptionName = ‘<subscription name>’              # UPDATE – This is the name of your subscription from the publishing file
    • $setupDir = "C:\BizTalkGurus"                                   # OPTIONAL UPDATE – Used for files on the remote and local servers
    • $basePathToScripts = ‘<path to scripts>’                    # OPTIONAL UPDATE – This is used if you run the scripts through an IDE
    • $workStorgeAccount = ’<update for auto configure>’    # OPTIONAL UPDATE – Update with the storage location of the helper files for auto configuration
    • $installCU2 = ‘false’                          #OPTIONAL UPDATE – Must upload BizTalk CU2 exe to the root of the Storage Account above
    • $installClassicShell = ‘false’              #OPTIONAL UPDATE – Must upload classic shell version 4.1 exe to the root of the Storage Account above
    • $installBizTalkProv = ‘false’               #OPTIONAL UPDATE – Must upload 5 files from BizTalk 2013 Dev box to the root of the Storage Account above
  • Run Master.ps1 in an elevated PowerShell window.  If running in an IDE it is best to run the script from the command windows rather than hitting Run. 

The end result will look something like this:

What is included

Prerequisites

  • Tested with PowerShell late- May release.  PowerShell versions before March (sorry can’t keep the versions straight) will fail in the VM creation because static IP Addresses are assigned to the VM’s in the network. 
  • Ensure you have a working PowerShell installation before running.  This can be tested by running Get-AzureLocation to ensure it returns values.  Details on how to set this up can be found here.
  • Scripts must be ran elevated. 

Main PowerShell Scripts

  • Master.ps1 – This is the only file you need to run.  Ensure you run from an elevated PowerShell window.  Takes a bit over an hour for full configuration.
  • variables.ps1  – This is the only file you need to update.  You should only need to look at the first 8 parameters. 
  • 00_Create_Network.ps1 – This script creates the affinity group, storage account, and virtual network.
  • 01_Create_PDC.ps1 – This script create the domain controller by promoting a windows server using remote PowerShell. It also creates all the groups and uses needed for the BizTalk configuration.
  • 02_Create_Servers.ps1 – This script creates the SQL and BizTalk Servers using the Enterprise Edition of the Microsoft provided images. 
  • 03_Configure_Servers_RemoteProp.ps1 – This script uses remote PowerShell to complete additional configuration on the SQL and BizTalk Servers.  If flags are set, files will be downloaded to the servers to complete configuration.
  • 04_Configure_Servers_LaunchRDP.ps1 – This script will launch the Remote Desktop sessions needed to complete the auto configuration.
  • 05_RemoteDesktop – Downloads the remote desktop connect for each server.  This is stored inside a new folder called RemoteDesktop in the scripts folder.
  • 99_RemoveVM.ps1 – End-to-end clean up script.  Removes all items created based on the variables file.  Note – due to limitations in Virtual Networks only run this if you have no other virtual networks on your subscriptions.   If you do, you should comment out the virtual network section of this script.
  • SaveMoney_StopAll.ps1 – Scripts to stop and deallocate all the Azure Virtual Machines based on the values in your variables file.
  • SpendMoney_StartAll.ps1 – Scripts to start all the Azure Virtual Machines based on the values in your variables file.
  • Configs folder – NetworkConfig.xml – This is the XML Configuration used to create the virtual network in Azure.
  • Helper Files – Helper files are only needed if you plan to auto-configure BizTalk, auto install CU2, or auto install Classic Shell
    • multinodeconfigDemo_DomainHosts.xml – XML file used by the BizTalk Provisioning tool to auto configure the BizTalk group.  No changes are needed to this file.
    • rdp.exe – Use to auto launch a Remote Desktop session to complete the BizTalk configuration. 
    • RunBizTalkTask_Domain.xml – File used to create a Windows Scheduled Task.
    • RunLocalClient.ps1 – PowerShell script to be ran locally on the First BizTalk Server in the group. 
    • RunLocalClientServer2.ps1 – PowerShell script to be ran locally on all Other BizTalk Server in the group. 
    • StartPowershell.exe – Helper exe to enable Windows Scheduled Task to easily run a PowerShell script in elevated mode.
    • Optional – ClassicShellSetup_4_1.exe – You need to be download from http://www.classicshell.net/ and uploaded to your container.   Ensure the version names match what is in the 03_Configure_Servers_RemoteProp.ps1 file.
    • Optional – BizTalk 2013 CU2 – This needs to be downloaded yourself and uploaded to your own public Azure container.
    • Optional – BizTalk.Provisioning.files- This needs to do downloaded yourself and uploaded to your own public Azure container.

To enable full end to end auto configuration do the following (Note – must be a MSDN users):

  • Put all helper files on your own public Azure storage account container
  • Update $workStorgeAccount in the variables file to the full address of your storage container
  • To enable auto install BizTalk 2013 CU2
    • Download BizTalk 2013 CU2 exe and upload it the root of your storage account
    • Update $installCU2 in the variables file to “true”
  • To enable auto BizTalk configuration
    • Create a BizTalk 2013 Development Virtual Machine in Microsoft Azure (script here)
    • Copy the 5 files in the C:\BizTalk_Provisiong folder
    • Upload the 5 files to the root of your storage account

If you do the auto configuration (witch is highly recommended for anyone who plans to run the scripts more than once) your end result will look like this.

Known Issues and Limitations:

  • At present the auto configuration is only possible if you download and host the BizTalk Provisioning files yourself.  This will change in the future and I will release a new set of scripts once the process is cleaner.
  • Every once in a while Azure is really slow.  I have added some speed breaks in the scripts to better account for this.
  • Every once in a while the auto configuration fails.  The servers get joined to the group by no hosts are created.  At that point, they should just be created manually or remove and restart all the scripts from scratch.
  • If you have an existing virtual network associated with your subscription the 99_RemoveVM script will delete it or fail trying.  This is a limitation in PowerShell. 

These scripts are over a year in the making!  Ok, ok, do not laugh.  I did not spend a year straight on them but I have spent over 300 hours trying to get them fully working end to end.  I have create no less than 100 domains while testing.  I never have an issue with the IasS side of things until the most recent release of PowerShell so things seem to be a little unstable at the moment,  All the issue I have ran into have been related to setting up the auto configuration.

Enjoy and I look forward to community feedback!

Windows Azure Virtual Machines – Stopped doesn’t equal Stopped Deallocated

Windows Azure Virtual Machines – Stopped doesn’t equal Stopped Deallocated

When driving down the street I think we all know what to do when we see a Stop Sign – we know we need to stop the car. 

What happens when you see Stopped as a status for a Windows Azure Virtual Machines?

A little over a month ago Microsoft announced they would no longer charge for a Stopped Virtual Machine.  This came as welcome news as I am sure many of you are like me – have left something on for a few days and received a bill for it. 

What Microsoft introduced was really a new definition of Stopped to go along with the existing process they had in place.

We now have the following two definitions:

  • Stopped
  • Stopped – Deallocated

When you create a Virtual Machine inside Windows Azure you get a Cloud Service to basically host that virtual machine role.  You can add more than one Virtual Machine to that Cloud Service and would do so to achieve high availability. 

Up until a few days ago, that Cloud Service was always hidden from the Windows Azure Management Portal.  You could see it when you deleted the last virtual machine in the service or using PowerShell.  Now, you can see the services inside the UI – so that should help with the confusion.  The Cloud Service, along with other things, is what gives your Virtual Machine a Virtual IP address (VIP). 

In the pictures below you can see a few of my Virtual Machines and Cloud Services in the Stopped and Stopped Deallocated state. 

What’s the difference between Stopped and Stopped– Deallocated? 

Stopped $$$ – When a virtual machine is stopped you are still being charged for the virtual machine.  You also have the Virtual IP (VIP) address reserved and when you start the machine back up again you will get the same VIP.  This is useful when you must have the same VIP. 

Stopped Deallocated – When a virtual machine is stopped deallocated you are no longer charged for the virtual machine.  The catch is, when you start it back up you might not get the same VIP back again.  You get a message about this when you stop the Virtual Machine.

For all the work I am doing right now, the Stopped Deallocated state works just fine for me. 

See below on how to get your Virtual Machine into a Stopped or Stopped Deallocated state.

How to Stop a Virtual Machine ($$$)

  • PowerShell
    • Stop-AzureVM -Name $VirtualMachineName -ServiceName $ServiceName –StayProvisioned (StayProvisioned  is new in PowerShell as of 06/13)
  • Selecting Shut Down while logged into the Virtual Machine
  • REST API
    • Example: https://management.core.windows.net/<subscription-id>/services/hostedservices/swt003-sql01/deployments/swt003-sql01/roleInstances/swt003-sql01/Operations

How to Stop – Deallocate a Virtual Machine (Money Saver!!!!)

  • PowerShell
    • Stop-AzureVM -Name $VirtualMachineName -ServiceName $ServiceName –Force (Note if you don’t use –Force you get an annoying confirmation popup)
  • Using the Windows Azure Management Portal
    • You will get a confirmation informing you the Virtual IP will be released

Please post if I have missed any other ways to stop a Virtual Machine.

How are you using Windows Azure Virtual Machines?  Does the Stopped Deallocated state work for you?

 

Help! I Cannot Attach My Azure Data Disk to my Windows Azure Virtual Machine

Windows Azure Virtual Machines are the new IaaS offering from Windows Azure that allows for Virtual Machines in the Cloud.  Using Windows Azure Virtual Machines you have the ability to add additional Data Disks in addition to your core Operating System Disk to your Virtual Machines.  The number of disks you can add depends on the size of your Virtual Machine and range from 1 for an Extra Small VM to 16 for an Extra Large VM. 

Data disks should be used for installing all your program files used on the Virtual Machines.  In general, the Cache level should be set to Read Only to ensure optimal disk performance.  Each data disk can get up to 500 IOPS (Input / Output Operations per Second).  So as you can see spreading things out over more Data Disks should be able to boost IO throughput.

This brings me to the main point of this post – how to add a Data Disk to a Virtual Machine.  To do so, select the Virtual Machine inside the Management Portal and hover over the Attach link on the bottom.  If you have a Disk available to be attached, you will see an option to Attach A Disk or Attach Empty Disk.  Use the Attach Empty Dish to create a new Data Disk.  If you already have an uploaded VHD and want to attach it as a Data Disk, you must first go to the DISKS tab and create a Disk for the VHD.

What can get confusing is WHY any existing disk might not be able to be added to a Virtual Machine.

I have found four reasons why an existing Disk cannot be added to a Virtual Machine and two of them are not very obvious using the Portal.

The reasons you cannot add an existing Disk include:

  • It is being used by another Virtual Machine
  • You have reached the limit of adding Disks to your Virtual Machine
  • The Disk is created in a different data center / location than your Virtual Machine (view this on the DISK tab)
  • The Disk is created using a different subscription than the Virtual Machine (view this on the DISK tab)

Have you found any other reasons you cannot attach an existing Disk?

Hope this helps someone stuck trying to add a Data Disk to a Windows Azure Virtual Machine.

TechEd 2013 Session – Microsoft BizTalk Server 2013 in Windows Azure IaaS

If you are planning to attend TechEd North America this June in New Orleans, LA and are looking for information on BizTalk Server running in Windows Azure IaaS then my session is for you!

With the general release of Windows Azure IaaS and BizTalk 2013 Pay-Per-Hour pricing yesterday I figured now is a great time to plug my session.  The abstract is below.

Abstract: This session looks at the architecture of three new integration infrastructure scenarios that are made possible with BizTalk Server 2013 on Windows Azure IaaS. These scenarios include: Small Scale All Cloud, Hybrid Cloud Burst, and Near-Infinite Scale All Cloud. This session walks through each scenario in detail and explains the best use for each, including a cost analysis. Demos are provided to show key aspects of each scenario.

I’m excited about the opportunity to get to speak again at TechEd.  My last TechEd presentation was in 2007 covering BizTalk Orchestrations. 

One thing to note about the speakers this year at TechEd, I see both a Stephen Thomas and a Steve Thomas.  How funny!  Make sure if you want to learn more about BizTalk you attend the Stephen Thomas session – although I’m sure the other Thomas’s sessions will be good as well.

Look forward to seeing you at TechEd! 

Windows Azure IaaS is Live and BizTalk Server 2013 Hourly Pricing Released

Today Microsoft released to General Availability the Infrastructure as a Service (IaaS) side of Windows Azure.  This includes Virtual Machines and Virtual Networking.  Read complete details of the release here.

This is a huge milestone for BizTalk’ers because with the GA release we now have available on a per-per-hour basis both BizTalk Server 2013 Standard and Enterprise Editions.  This opens up a whole new spectrum of scenarios using this per-per-use model. 

We now have 3 new pre-built images for BizTalk 2013 available in the Image Gallery.  They include a BizTalk Evaluation, BizTalk Standard, and BizTalk Enterprise editions.  As best I can tell, the BizTalk Evaluation is just charged at the base rate of the VM.  The other two editions are more expensive.  They are priced as shown in the table below but they include the license to BizTalk.

COMPUTE INSTANCE NAME

VIRTUAL CORES

RAM

STANDARD

ENTERPRISE

Small (A1)

1

1.75 GB

$0.75/hr
(~$558/month)

$2.99/hr
(~$2,225/month)

Medium (A2)

2

3.5 GB

$0.84/hr
(~$625/month)

$3.08/hr
(~$2,292/month)

Large (A3)

4

7 GB

$1.02/hr
(~$759/month)

$3.26/hr
(~$2,425/month)

Extra Large (A4)

8

14 GB

$2.04/hr
(~$1,518/month)

$6.52/hr
(~$4,851/month)

COMPUTE INSTANCE NAME

VIRTUAL CORES

RAM

STANDARD

ENTERPRISE

A6

4

28 GB

$1.68/hr
(~$1,250/month)

$3.92/hr
(~$2,916/month)

A7

8

56 GB

$3.36/hr
(~$2,500/month)

$7.84/hr
(~$5,833/month)

Pricing current as of 4-16-13 per the Azure Pricing Page.

I have already played around with the Enterprise Edition running as a Small VM.  I would say, don’t even bother with this size.  If you want to give it a run, go right to the Large.  The few cents I saved per hour was more than eaten up with latency and overall general slowness of the small VM instance.  Giving the few cents price difference between Small and Large, I’ve surprised Small and Medium are even an option.  I also do not know how this pricing relates to any free usage you might get in the 90 day trial or MSDN.

CRITICAL – please remember you are charged for a VM even when it is STOPPED.  So if you create a Large BizTalk Enterprise VM you are charged until you DELETE the VM.

Let me know if you have played around with these new VM’s and what you think of them!  More to come on BizTalk and IaaS in the upcoming months.

Windows Azure IaaS and Creating a New Azure Virtual Machine Video

I have put together a quick 10 minute video that introduced Windows Azure IaaS and walks you through creating your first Windows Azure Virtual Machine! 

I put a lot of time into adding cool animations and fun dialog, or so I think.

[View:http://www.youtube.com/watch?v=zeNb6LDvVzQ]

Download this video from BizTalkGurus.com or view on YouTube.

Enjoy and let me know what you think!

Windows Azure Virtual Machine and Virtual Hard Disk Management Tool – Can save you Money!

Did you know you are charged for a Windows Azure Virtual Machine every hour it is deployed even if it is not running?  I would guess many people are not using their Virtual Machines at night when they sleep so shouldn’t we have a simple way to take them offline?  Wouldn’t it be nice to have a simple GUI interface to view Virtual Machines (VM) and Virtual Hard Disks (VHD)?

This tool solves all those problems!  This tool can be used to easily reduce your monthly Windows Azure compute usage and SAVE YOU MONEY!

I am pleased to announce the release of a community project called the Windows Azure Virtual Machine and Virtual Hard Disk Management Tool.  I have put over 300 hours into researching, building, and testing this tool.

Download Link: Windows Azure Virtual Machine and Virtual Hard Disk Management Tool

Read the current list of Known Issues: Known Issues 
The list of issues currently relates to Virtual Machines, Disks, and Virtual Networks that were created more than a few months ago.

The purpose of this tool is to make working with Windows Azure Virtual Machines and Virtual Hard Disk easier to use, easier to understand, and more cost effective.  Take a look at the feature list below. 

Sure – things like taking a VM offline can be done via PowerShell, but I have made it so easy using a simple WinForm even my mom could take a VM offline (if she only knew what offline meant)!  Better yet, setting up and configuring PowerShell for Azure is not required to run this tool. All you need is .Net 4.0 or higher, a management certificate installed in Azure, and your subscription ID.

Features:

  • Virtual Machines
    • View the Service Name, Deployment Name, Virtual Machine Name, and Location of online Virtual Machines
    • Take a Virtual Machine offline to save on compute time charges (they are stored on your local file system as XML)
    • Bring a local Virtual Machine online to use it again
  • Virtual Hard Disks
    • View all the VHD’s in the Source and Destination storage account container
    • Copy a VHD from one account to another account, even in a different data center
    • View the Lease status of VHD’s in the Source or Destination storage account
    • Break the Lease of a VHD in the Source or Destination storage account (for orphaned VHDs)
    • Delete a VHD in the Source or Destination storage account
    • Download a VHD in the Source storage account (this is slow – do not close the form)
    • View all the Disks and Disk Details associated with a single Azure subscription
    • View all the Images and Image Details associated with a single Azure subscription
    • Prevents you from making a copy of a vhd with an existing Lease

See the download link or ReadMe.txt for more details and basic How To’s.

Please keep in mind this is a free tool without support or warranty of any kind.  In the event of “bad things happening” I store a backup of the raw exported XML in the local folder.  I also do not touch the underlying VHDs when taking a VM offline.  Worse case, the VMs would need to be recreated using the Windows Azure portal.   

If you run into any issues, please use the Email Me link inside the form to let me know so I can research the issue.  I want this to be the best tool possible.

Stay current with the latest information on Windows Azure Virtual Machines Management by joining our Windows Azure Virtual Machine and Virtual Hard Disk Management Newsletter.

To use this tool you need to setup on values in the App.Config file. 

  <!-- Enter the Source Storage Account Details here. -->
 <add key="SourceStorageConnection" value="DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" />
 
  <!-- Uncomment and Enter the Destination Storage Account Details here. To work with only one account leave commented out.  
      To copy to the same store set this to the same as Source Account above. -->
  <add key="DestinationStorageConnection" value="DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" />
 
  <!-- Select to enable REST API Advanced Features - Subscription ID and Certificate are Required for this feature.  true | false -->
  <add key='EnableRESTApi' value='true' />
    
  <!-- Enter the full path to the -->
  <add key='CertificatePath' value='--Required for REST API not for File Copy--'/>
 
  <!-- Your Windows Azure Subscription ID -->
  <add key='SourceSubscriptionID' value='--Required for REST API not for File Copy--' />
 
  <!-- Local folder for exported VMs -->
  <add key='LocalVMLocation' value='--Required for REST API not for File Copy--' />

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

Basic Steps to take a Virtual Machine Offline Note – the Disk and VHD are not touched in this process

  1. Ensure the 4 REST related configuration values are set in the App.config file
  2. Make sure you have clicked Start Query on the Manage and Copy VHD tab
  3. On the Manage Virtual Machines, select the VMs you want to take offline
  4. Click Take Selected VMs Offline
  5. Process will run in more than one thread unless you have more than one operation on the same service, those will queue
  6. Output is stored on the file system in the Local VM Location defined in the App.config
  7. Watch for Status and Error messages in the Status window

Basic Steps to take a Virtual Machine Online Note – the Disk and VHD are not touched in this process

  1. Launch the tool and click on the Manage Virtual Machine tab
  2. Select the VMs you want to bring online
  3. Click Restore Selected VMs
  4. Process will run in more than one thread unless you have more than one operation on the same service, those will queue
  5. Watch for Status and Error messages in the Status window

Download Link: Windows Azure Virtual Machine and Virtual Hard Disk Management Tool

Tricks for Deleting Windows Azure Virtual Machines via the REST API

While I call this post “tricks” for deleting a Windows Azure Virtual Machine it really is not a trick.  It does require a little deeper understanding of how Windows Azure creates and hosts Virtual Machines in order to use the REST API to delete them.

Let us start with some background.  When you use the Windows Azure Portal to create a new Virtual Machine, a hidden service is created for you that will host the virtual machine role.  This will allow more than one Role (i.e. Virtual Machine) to be connected to this service to allow for the automatic round-robin of calls made to the exposed ports.

Below is a list of the Services in one of my Azure Account.

Using the REST API to list all Services, I see that not all of the services returned are shown on the screen in the portal.  The service named Test0206D is not listed inside the portal.

  <HostedService>
    <Url>https://management.core.windows.net/<subscription-id>/services/hostedservices/Test0206D</Url>
    <ServiceName>Test0206D</ServiceName>
    <HostedServiceProperties>
      <Description>Implicitly created hosted service2013-02-07 01:20</Description>
      <Location>West US</Location>
      <Label>VGVzdDAyMqZE</Label>
      <Status>Created</Status>
      <DateCreated>2013-02-07T01:20:58Z</DateCreated>
      <DateLastModified>2013-02-07T01:21:32Z</DateLastModified>
      <ExtendedProperties />
    </HostedServiceProperties>
  </HostedService>

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

In this case the Description clearly tells us this was created for us and when.  When we delete a Virtual Machine, the service will start to show up inside the portal.  If you never want to deploy a new Virtual Machine to that service again, it can be deleted.  Otherwise, keeping the service allows the DNS Name to be reserved and allows a new Virtual Machine Role to be added to this service at a later point. 

In order to Add a new Role (i.e. Virtual Machine) to an existing service, use the From Gallery option to create the Virtual Machine. Then, on step 4, Virtual Machine mode, select the Connect To An Existing Virtual Machine option. You will get a drop down of available options.

This background leads to the reason it might be tricky to delete a Virtual Machine using the REST API.

To Delete a stand-alone Virtual Machine

In an earlier blog post I listed the way to delete a Virtual Machine was to do an HTTP DELETE to the following URL.

https://management.core.windows.net/<Subscription-ID>/services/hostedservices/<Service-Name>/deploymentslots/Production

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

While this work, it only works in the case of a strand-alone Virtual Machine.  If you do not know if the Virtual Machine is stand-along or attached, do an HTTP GET to the URL above.  If you only have one <Role> element than it is a stand-alone Virtual Machine.

To Delete a Virtual Machine that is attached to another Virtual Machine

In order to Delete a Virtual Machine that is part of another service, you need to Remove the PersistentVMRole.  This is done by an HTTP DELETE to the following URL.

https://management.core.windows.net/<subscription-id>/services/hostedservices/<service-name>/deployments/<deployment-name>/roles/<vm-name>

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

The service name and virtual machine name are straight forward but the deployment name is tricky.  The deployment name is usually the Service Name but it can also be the name of the first virtual machine you put into the service.  To find out for sure, do an HTTP GET to the following URL to get the details of the service deployment.

https://management.core.windows.net/<subscription-id>/services/hostedservices/<service-name>/deploymentslots/Production

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

This will return an XML response that looks like the following.

<Deployment xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Name>Test0202A</Name>
  <DeploymentSlot>Production</DeploymentSlot>
  ...removed...
</Deployment>

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

The Name element is the name of the Deployment. 

In summary, a single call can be made to delete a stand-alone Virtual Machine.  For scenarios that join several Virtual Machines together inside a single service it takes a deeper understanding of that service’s properties in order to remove a Virtual Machine from that service.

Looking for a simple tool to work with the Windows Azure REST APIs?  I have a free tool here that greatly simplifies making REST calls.