Exploring the Service Bus Relay, Queues,Topics and Subscriptions with Visual Studio 2012 RC and Windows Azure SDK 1.7

Exploring the Service Bus Relay, Queues,Topics and Subscriptions with Visual Studio 2012 RC and Windows Azure SDK 1.7

 

Introduction

In my post on Future BizTalk Server Release and Microsoft Integration Strategy I discussed the three pronged Microsoft integration strategy:

  • On-premise BizTalk (a CTP of next version will be released this summer)
  • BizTalk IaaS (Infrastructure as a Services), Windows Azure offers services like Virtual Machine and Virtual Network to enable to have your BizTalk infrastructure in the cloud;
  • BizTalk PaaS (Platform as a Services) with EDI/EAI integration capabilities using the Service Bus.

In this post I like to share my exploration of the Windows Azure Service Bus. My focus is on BizTalk professionals that do not yet have much experience/knowledge of the Windows Azure Service Bus.Two sessions by Clemens Vasters from last TechEd North America were my baseline before I started my exploration:

  • Overview and Roadmap of Windows Azure Service Bus
  • Achieving Enterprise Integration Patterns with Windows Azure Service Bus

I will explore capabilities of Queues, Topics and the Relay as depicted in overview below of the Service Bus.

image

In picture above you see different solutions, devices that through services bus communicate with Line of Business applications (SAP, Oracle E-Business Suite), SharePoint and many other systems and applications.

The Windows Azure Service Bus offers capabilities that allow you to create solutions in the cloud that are based on message-oriented-middleware technologies including reliable message queuing and durable publish/subscribe messaging. Ergo its is a messaging infrastructure in the cloud. This means that Service Bus can act as a broker for asynchronous messaging (decoupling). Therefor this infrastructure can support publish-subscribe, temporal decoupling, and load balancing scenarios similar to BizTalk Server. Queues, Topics and Subscriptions are services (entities) of the Service Bus that provide durable, asynchronous messaging.

Relay is an entity that allows you to pass-through message (relay) from one system/application to another. Relay service that sits in Service Bus brings the ability to securely expose Windows Communication Foundation (WCF) services (even orchestrations or LoB Services) that reside within your enterprise network to the public cloud, without having to open up a firewall connection or requiring intrusive changes to your enterprise network infrastructure.

Note: Notifications topics and bridges are services (entities) in Service Bus that will be available later and are currently not present.

Get started

To get started I created a new VM with Windows Server 2008 R2 with Visual Studio 2012 RC and Windows Azure SDK 1.7. Now the SDK can also be used with Visual Studio 2010 SP1. You can use either VS2010 or VS2012 with this SDK.

Samples

The sample on the relay is a 101 sample on using the Relay. The other sample is more a walkthrough, using Queues, Topics and Subscriptions in the service bus by using the Service Bus Explorer. There illustrative to show the capabilities of these services if you are not familiar with them. You’ll need either Visual Studio 2010 SP1 or Visual Studio 2012 RC, Windows Azure SDK 1.7 (June 2012) and a namespace in Service Bus. Relay samples in this post and the walkthrough with the Service Bus Explorer is done in Visual Studio 2012 RC.

Create a namespace

To create a service namespace:

  • Log on to the Windows Azure Management Portal. If you end up in the new portal then select preview and choose Take me to the previous portal (Managing the Service Bus is still done in the old portal).
  • In the lower left navigation pane of the Management Portal, click Service Bus, Access Control & Caching.
  • In the upper left pane of the Management Portal, click the Service Bus node, then click the New button.
  • In the Create a new Service Namespace dialog box, enter a Namespace, and then to make sure that it is unique, click the Check Availability button.
  • After making sure the namespace name is available, choose the country or region in which your namespace should be hosted (make sure you use the same Country/Region in which you are deploying your compute resources), and then click the Create Namespace button.
  • The namespace you created will then appear in the Management Portal and takes a moment to activate. Wait until the status is Active before continuing.

Service Bus NuGet Package

The Service Bus NuGet package is the easiest way to get the Service Bus API and to configure your application with all of the Service Bus dependencies. The NuGet Visual Studio extension makes it easy to install and update libraries and tools in Visual Studio and Visual Web Developer. The Service Bus NuGet package is the easiest way to get the Service Bus API and to configure your application with all of the Service Bus dependencies.
To install the NuGet package in your application, do the following:

  • In Solution Explorer, right-click References, then click Manage NuGet Packages.

image

  • Search for “WindowsAzure.ServiceBus” and select the Windows Azure Service Bus item. Click Install to complete the installation, then close this dialog.

image

Building a SIMPLE Relay Sample

To build a sample that will make use of relay service you need to create a WCF-service. The following steps show how to build a very basic service that can be exposed in the Service Bus.

  • Within Visual Studio, create a console application
  • Add the Windows Azure Service Bus NuGet package to the projects. This adds all of the necessary assembly references to your projects.
  • Add a class to project and give it a descriptive name.
  • Rename the class to HelloRelayContract and then implement code like below:

image

  • Add a new class to the project and rename it HelloRelay and then implement code like below:

image

  • In program.cs implement the following code:

image

  • In order to perform management operations, such as creating a relay connection, on the new namespace, you need to obtain the management credentials for the namespace.
  • Select the namespace you just created from the list show below:

image

  • The Properties pane on the right side will list the properties for the new namespace:

image

  • The Default Key is hidden. Click the View button to display the security credentials:

image

  • Open app.config and make the appropriate adjustments like below:

image

  • You can past the default key in the issuerSecret attribute.
  • Now that you have build and configured the service you can run the console application that will act as host for the service.

image

  • Copy the address in your browser and you will get result like below:

image

Service Bus Explorer

One of new tools available through the SDK is the Service Bus Explorer. With the SDK installed you have access to several Azure features from Server Explorer.
image
To access the service bus through the Server Explorer:

  • Right click the Windows Azure Service Bus.
  • Select add new connection.
  • Fill in the namespace, Issuer Name and Issuer Key.

image

  • Click Ok
  • You now have access to Service Bus

image

QUEUES, TOPICS AND SUBSCRIPTIONS USING THE SERVICE BUS EXPLORER

To build a sample that will make use of queues, topics and subscriptions. These services support brokered messages. With the service explorer I can manage the service bus without using the portal (see previous paragraph). You can create queues through code or through using the explorer. Using the explorer you can right click Queues and add a new Queue.

image

Fill in a name, change some the values if necessary, click options and click Ok. You will then see your queue in Service Explorer.

image

You can right click the created queue and select Send Test a Message. You will then see dialog pop up with the message.

image

You will notice in the properties of the queue that the message count is 1.

image

With topics you can do the same. Right click topics and select Create New Topic.

image

Fill in a name, change some the values if necessary, click options and click Ok. You will then see your topic in Service Explorer.

image

Before you can send a message to the topic you need to create a subscription. Select subscriptions and right click. Select Create New Subscription .

image

Fill in a name, change some the values if necessary, click options and click Ok. You will then see your topic in Service Explorer.

image

Like queues you can send a test message through the topic.You can right click the created topic and select Send Test a Message. You will then see dialog pop up with the message. In subscription properties you will see message count is 1.

image

On the subscription you can right click and select Receive Message. A dialog will pop up with the message.

image

Now you can create more subscriptions in the topic and then sent a test message. You will then see that both subscriptions in the topic will have that message. Now you can use rules to determine, which subscription will get which message. This is new functionality with topics and subscriptions. You can within a subscription add a rule. Open the desired subscription and right click rules. Select Create New Rule. A dialog will appear.

image

Give it a name and add a Sql Filter Expression. Click Ok. You now have created a rule on the subscription.

Note: The Overview and Roadmap of Windows Azure Service Bus session of Teched NA will provide more context on Service Bus explorer. It will also demo using queues and topics with new Worker Role with Service Bus template.

Call to action

To learn more besides the two sessions from TechEd I suggest to look at Developing Applications that Use the Service Bus on MSDN, Alan Smith’s Developers Guide to AppFabric and resources listed below:

Update: Thanks to Paolo Salvatori I have been pointed some of the articles and samples he wrote on the Service Bus:

  • Service Bus Explorer (Windows Azure Sample)
  • How to integrate a BizTalk Server application with Service Bus Queues and Topics
  • How to integrate a WCF Workflow Service with Service Bus Queues and Topics

I myself got a little more feeling now with the new SDK and some of its features. My experience with the Service Bus explorer within Visual Studio 2012 RC so far has been great. It works seamlessly and is straight forward. Enjoy your own exploration!

Cheers,

– Steef-Jan

Microsoft TechEd North America 2012-Day 4

So this post is a little delayed due to all of the excitement around the BizTalk sessions.  However the sessions were that good that I wanted to still publish the post.

Azure Service Bus Solution Patterns  -Clemens Vasters and Abhishek Lal

Another session by Clemens and Abhishek.  This time around it was a very practical session based upon some Customer Use Cases and how to implement some popular integration design patterns based upon the “Integration Bible” – Enterprise Integration Patterns.  To view the actual session on Channel9, click here.

Some of the Use Cases included:

  • Web Services For Retailers
    • Company from Italy
    • Provide SAAS solution for Retail Stores
    • Seed local retail outlets with Catalogue and Pricing information
    • Push out to retail stores
      • Use Topics to distribute information to each retail store
  • SaaS with Dynamic Compute Workload
    • High Performance Computing (HPC) scenario
    • Command and Control messages sent in from Service Bus
    • ISV specialized dynamic compute capacity provider
  • Consumer Web Site
    • Web site that searches for data about people – credit check, criminal check etc.
    • Their challenge was back end data co-ordination
    • Different profiles for users who have different access to to back end services
    • Queues for decoupling the web layer from middle-tier services

 

Scaling things out

Next Clemens walked us through a scenario that Microsoft has been working on with a particular customer.  The solution was related to remote controlling air conditioners.  The idea is that a consumer would have the ability to manually control it but also power providers could *potentially* control it to prevent rolling brown-outs from occurring.  Instead of instituting  wide spread rolling brown-outs, each customer could alter their consumption. Collectively these savings add up and prevent demand from exceeding supply.  I am a little skeptical about a power company(I work for one) controlling someone’s air conditioner but in theory it makes a lot of sense.

The requirements for this solution includes:

  • Pair devices, such as air conditioner, to local Wi-Fi connection
  • Users need the ability control the device
    • Control requests could be made from back yard or across the world
    • Service Bus makes these control requests possible from anywhere that has an internet connection.
  • Devices will then send consumption data to Azure where the data can be viewed on a mobile device. This data will make its way to Azure via Service Bus.  The premise behind this is if customers are more aware of their consumption patterns, then they may try to alter them.  This is something that my organization has also been investigating.

So a question remains, these types of consumer devices will not have the .Net Service Bus bindings installed so how will they actually communicate?  The answer is really HTTP.  You can send HTTP requests to the Service Bus and in this case Clemens introduced a concept that he likes to call “N-HTTP”.  It is a similar to the “NoSQL” movement but in this case is related to HTTP.  HTTP in many cases includes HTTP Headers but also an entity body.  The entity body could include JSON content, XML content etc.  The challenge with entity bodies is that you need a parser to package the information up in requests or un-package it when receiving responses.  This would further complicate things as these parsers would need to be loaded into these consumer devices.  What’s interesting is HTTP Headers is they are well understood, across devices, systems, technology stacks etc,  and do not require parsers.  So if you can get away with sending key/value pairs when sending or receiving messages then this solution should work for you.

Receiving messages from Service Bus generally includes using ’long polling’ when waiting for messages.  Using  long polling sockets isn’t a great use of power resources for devices that do not have permanent power sources (devices that rely on batteries).  With this in mind, Microsoft has been working with other industry leaders on the AMQP (Advanced Message Queuing Protocol).  AMQP is a popular queuing technology that is used in financial brokerage settings.  Another benefit of using AMQP is that it has a quieter socket profile which results in lower battery consumption.  So this is an area that Microsoft is investing in that will have wide spread benefits.Cool Stuff!!!

 

Message Channel Patterns

Abhishek was back on point and walked us through some popular messaging patterns including:

  • Pub-Sub
    • Accomplished via Topics
  • Content Based Router
    • Using Topics based upon a Subscription Rule
  • Recipient List
    • Sender wants to send the message to a list of recipients
    • Common use-cases
      • Order processing systems – route to specific vendors/departments
      • “Address LIKE ’%First%’
  • Message Routing
    • Session re-sequencer – receiving messages out of order and then using the defer method to postpone processing the next message until you receive the next message that is “in order”

I must admit, when I learn more about the Service Bus I do get a little giddy.  I just see it as such an enabling technology.  It facilities building applications that just wouldn’t be possible or cost prohibitive in years gone by.  Whether it is submitting SAP  timesheets remotely or reporting customer power outages it is an amazing technology the opportunities are endless when it comes to bridging data center boundaries.

 

Mobile + Cloud: Building Mobile Applications with Windows Azure – Wade Wagner

Wade Wagner, a former Microsoft Azure Evangelist, put together a pretty interesting session related to Windows Phone and Azure.  To watch this session on Channel 9 click here.  In the past I have followed some of the work he did with the Mobile toolkits for the different mobile platforms, but just haven’t had the time to take a closer look.

This session focused primarily on Windows Phone 7 and how it interacts with some of the Azure services (Storage, SQL Azure, Tables, ACS).  Personally, I think these technologies complement each other very well.  Especially in the area of bridging mobile devices with on-premise LOB solutions and leveraging the Access Control Service (ACS) for authentication.

Three reasons for Device + Cloud

  • Allows for new application scenarios
  • The cloud levels the playing field
  • The cloud provides a way to reach across device platforms and a larger pool of resources from which to pull

Why Azure?

  • PaaS you build it, Windows Azure runs it
  • Automatic O/S patching
  • Elasticity and Scale
  • Utility Billing
  • Higher-level services
  • ACS, Caching, CDN (cache static content), Traffic Manager (route traffic across Azure datacenters based on locale)

Wade then demonstrated a scenario really lends itself well to this technology.  A mobile application that will take advantage of Social Identity providers(Windows Live, Google, Yahoo) for authentication via the Access Control Service.  Wade demonstrated that this isn’t as complicated as it sounds.  With the help of a Nuget package and adding a STS reference we can get this working in the matter of a minutes.  Wade then added some additional functionality to consume a ASP.Net Web API.  Most presenters would have left their demo there.  Giving people the information to build the services but then leaving out some “real world” gaps around security.  Wade did take his demo one step further and then showed we can use the ACS service to authorize user requests as well.  Before the ASP.Net WebAPI method is called, we can intercept this request and validate that the token that has been included as part of the HTTP Request is a valid ACS token.  Provide the token is valid, the appropriate data will be returned.

Wade then wrapped up his session demonstrating how we can use the Azure Push Notification service to serve up “toast notifications”.  Another set of useful information that I hope to play with soon. 

If you are into mobile apps, you definitely owe it to you to watch this session so you can learn about all of the Azure services that you and your customers can benefit from.

Comparing Cloud Server Creation in Windows Azure and Tier 3 Cloud Platform

Comparing Cloud Server Creation in Windows Azure and Tier 3 Cloud Platform

Just because I work for Tier 3 now doesn’t mean that I’ll stop playing around with all sorts of technology and do nothing but write about my company’s products. Far from it. Microsoft has made a lot of recent updates to their stack, and I closely followed the just-concluded US TechEd conference which covered all […]
Blog Post by: Richard Seroter

Resolving ’Invalid RSS MIME type: text/xml’ when crawling RSS Data with the FAST Web Crawler

Overview
When we were attempting to crawl an RSS feed for a client using the FAST Web Crawler, we found the following error in our RSS Crawl Logs (located at \FASTSearch\var\log\crawler\node\fetch\<content collection name>)
“Invalid RSS MIME type: text/xml”

We found that the MIME type being indicated by the server was indeed “text/xml”.  However, we were able to crawl […]
Blog Post by: Michael Gerety

Forcing FAST Web Crawler to Re-Crawl Data Immediately

Overview
While attempting to set up the FAST Web Crawler for a client, I was unable to find a way to force FAST to re-crawl data immediately.  The web crawler uses an internal scheduler to schedule fetch operations.
There are a large number of options for the crawleradmin.exe command line tool that looked like they might work:
(from […]
Blog Post by: Michael Gerety

TechEd 2012: BizTalk Future & Azure Service Bus

I Just returned from TechEd 2012 and one of my favorite sessions was ‘Application Integration Futures – The Road Map and What’s Next in Windows Azure’. It was refreshing to see the first slide’s title “Key takeaways: we are innovating in BizTalk”.
Here is the abbreviated messagefrom the slide deck:

Continued investment in BizTalk – take your […]
Blog Post by: Stan Kennedy