Windows Azure Virtual Hard Disk (blob storage) Cross Account Copy, Lease Break, and Management Tool - Stephen W. Thomas BizTalk MVP - BizTalk Blogs

Recently the Windows Azure Storage API’s were updated to include the ability to do cross account, asynchronous blob transfers. This means it was no longer necessary to download the blob in order to upload it to a different Windows Azure account. You can read all about this here.

Over the past few weeks I have been working with the Windows Azure Virtual Machine feature that is currently in preview mode. I talked in a past blog post and video about using the Gallery to create a BizTalk 2010 R2 CTP Virtual Machine.

I am working on proving out and understanding the scenario of creating a Master Image (or Master VHD for non-domain Virtual Machines) and sharing it with a large team of developers. Why? Because on past projects we have spent considerable time and resources (i.e. money) creating, maintaining, and provisioning developer images. For example on a recent project an urgent issue arose and we needed a dedicated BizTalk environment to test a cumulative update release. It was requested on Friday morning. Still today on Wednesday it was not created (4 business days later). With Windows Azure Virtual Machines and a pre-built image this would literally be a few minute task and something I could do myself without the involvement of 3 to 5 other people. In total, so much time (i.e. money) can now be saved with Windows Azure Virtual Machines!

This type of example leads me to my tool and why I created it. In my scenario, a Windows Azure Virtual Machine guru would create a master image or vhd for distribution. With the new asynchronous cross account copy, sending an Image or VHD to a new developer is a snap… as long as you know the “rules”. I created, copied, deleted, and destroyed over 100 Virtual Machines at a total cost of around $5.00 across multiple Azure accounts to fully understand how to successfully copy an Image and VHD from one account to another without issues. I put all that into a simple tool that anyone can use to manage vhds and copy them from one account to another. See below for known issues, features of the tool, and quick how to’s.

I welcome all feedback! Please note this tool is an excellent example of why I am NOT a Windows Form guy and in no way should anything done in this tool be considered a best practice. This tool is more about getting the job done and trying to make things easy to understand.

Download: Windows Azure VHD Helper Tool

Basic Steps for Cross Account VHD Copy:

Shut down your Virtual Machine.
Delete your Virtual Machine.
Delete the associated Image or Disk inside the Virtual Machine Portal. This does not remove the underlying blob of the vhd.
Copy the blob from one storage account to another.
Add your vhd back and recreate the virtual machine.

Issues (per my research) with VHD Copying as a Preview Feature– make sure you understand these issues and limitations. I assume most are related to preview issues and this process will get simpler with time.

If you copy a vhd that is Leased (configured as an Image or Disk inside the Virtual Machine Portal) the Source Disk will become unusable.
If you copy a vhd that is Leased (configured as an Image or Disk inside the Virtual Machine Portal) the Destination Disk will likely not work. I would say about 75% of the time they do not work. I just re-tested this tonight and they worked.
Copying from one data center to another seems to be as slow as downloading the file.
When downloading the file, make sure you do not close the form or the download will stop.
Copy Status does not seem to be working. I either get NULL or Pending all the time. Probably a user error on my part…. The best way to find out if the copy is complete is to check the file size on the destination. It is 0 MB until the copy is complete.

Features:

Visual view of vhds contained inside a Storage Account
Simple, one click Delete of vhds
Simple, one click Lease Break of vhds (for orphaned vhds – see more here)
One click download of a vhd, tested to work with Hyper-V (downloads at a rate of about 3 MB/sec)
Support for easily making a copy of a VHD within the same Azure account
Support for making a vhd copy across Window Azure Accounts within the same region within a few seconds
Support for making a vhd copy to any account in any region (slow when out of region)
Prevents you from making a copy of a vhd with an existing Lease
Uses the Windows Azure Storage API version 1.7.1 (github)

Advanced Features (with REST API Management Certificate):

Visual view of all vhds contained inside a single Windows Azure Account
See a “***” by vhds inside the Source storage account
List all custom user images inside a single Windows Azure Account

Missing Features (i.e. items I hope to add later):

Ability to upload a vhd
Ability to get details on an Image

Quick How-To’s:

How to work with just one Storage Account:

Set just a Source Configuration inside the app.config file. I also have a check box for “source only” that will try to auto determine what you want to do. The default is Source Only.

How to Copy a vhd into the same account:

Inside the app.config file for the tool, set the Source and Destination to the same Storage Account.

How to Copy a vhd into another account in the same region (inter-account or cross-account):

Inside the app.config file for the tool, set the Source to the Storage Account that has the source vhd and set Destination to Storage Account that you want for the Destination. The destination container vhds does not need to exist. The tool will create it for you. Click “Copy”. The Destination list should refresh and the copy already be compete. You will know it is complete when the size of the vhd is not 0.

How to Copy a vhd cross region:

Inside the app.config file for the tool, set the Source to the Storage Account that has the source VHD and set Destination to Storage Account that you want for the Destination. The destination container vhds does not need to exist. The tool will create it for you. Select a vhd in the List box that is not lease. Change the vhd name if you wish. Click “Copy”. Click on “List All VHDS” on the top right until you see the file size of the Destination change from 0 to the size of the VHD. This will take a long time, up to 8 hours. I would not close the form until I have confirmed it was completed.

How to Download a Source vhd:

Inside the app.config file for the tool, set the Source to the Storage Account that has the source VHD. Click on “List All VHDS”. Select the VHD you want to download inside the List Box. Click on “Download”. You will be prompted for the download location. Make sure you have enough disk space. Leave the tool open until you see a Completed notice on the top right of the tool. This will take up to 8 hours. Do not close the form or the download will stop.

How to Break a Lease or Delete a vhd on the Source or Destination:

Inside the app.config file for the tool, set the Source and Destination to the Storage Accounts that has the VHDS. Click on “List All VHDS”. Using the List Box, select a single vhd from the list. Click on “Break VHD Lease” to break the existing lease or “Delete VHD” to delete the vhd. Note: this operations are not undoable. Using these incorrectly can result in loss of data or vhds.

I would like to thank the following references:
- How not to throw an exception using CreateIfNotExists on a container - http://blog.smarx.com/posts/testing-existence-of-a-windows-azure-blob
- How to do an Azure REST API Service call - http://msdn.microsoft.com/en-us/library/windowsazure/ee460782.aspx
- How to have a 2nd thread update the main windows form - http://stackoverflow.com/questions/10775367/cross-thread-operation-not-valid-control-textbox1-accessed-from-a-thread-othe
- Async across copy with SDK 1.7.1 - http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-asynchronous-cross-account-copy-blob.aspx

Download: Windows Azure VHD Helper Tool