Like some of you I have been experimenting with the Service Bus Queues, Topics and subscriptions. The information from Microsoft gives you a straight forward insight how to send messages to queues, topics and subscriptions using .NET.  The first step to interact with the Service Bus Queues, Topics and subscriptions is by creating an account with Windows Azure through a trail or buy option. The next step is creating a service namespace through the Windows Azure Portal followed by obtaining the management credentials you need when setting up Service Bus Connection string.  To create a solution that can interact with the Service Bus you need to have a reference in your Visual Studio project to the Microsoft.ServiceBus.dll on your machine, which is a part of the Windows Azure SDK. In your code template (for instance C# form) you will add two using statements:  
  • using Microsoft.ServiceBus;
  • using Microsoft.ServiceBus.Messaging;
Now everything is setup for coding your interaction with the Service Bus.  In this post I will show my sample I have built that sends messages to the queue. These messages are then read by the SB-Messaging Adapter from BizTalk Server 2010 R2 CTP. BizTalk Server is on a VM in the cloud (see my post on BizTalk Virtual Machine in Windows Azure using Quick Create and BizTalk Virtual Machine in Windows Azure using Quick Create–Part II). These messages are then stored in SQL Server 2008 R2 table using the BizTalk Server 2010 R2 CTP Adapter Pack WCF-SQL Adapter. Below you will see the diagram of the solution.

image

The client application is a .NET Windows Form application that will send messages to a queue in Windows Azure Service Bus. An application in BizTalk Server 2010 R2 has a port with a receive location configured with SB-Messaging Adapter. With the port a mapping will be performed to Insert Operation Message format. A send port has a subscription on messages with the SQL Insert format. The send port will send the messages to SQL Server 2008 R2 resulting in inserts in a SQL Table.

The client application

The client application is a Windows Form application. It has one form, where details can be filled for a runner like Name, number and time. I use the participants textbox to simulate a load to the Service Bus (see the latency section).  image
When I hit submit after filling in participants (i.e. 1), name, number and time the runner information will be sent a message to the Service Bus Queue.  image
The Service Bus Queue

Service Bus queue can be created through code, the service bus explorer within Visual Studio 2012, the Service Bus Explorer created by Paolo Salvatori (see my post Visual Studio Service Bus Explorer versus Standalone Service Bus Explorer) or through Windows Azure Portal.

image

BizTalk Server 2010 R2 Solution

The BizTalk Server solution consists of a schema of incoming messages from the Service Bus (runner message), a map and schema’s generated through Adapter Metadata Utility (i.e. Consume Adapter Service Add-in in Visual Studio 2012, see also TechNet Wiki article on Adapter Pack). Below you can see the artifacts created in a BizTalk project in Visual Studio 2012

Runner Schema

image

SQL Insert Operation Schema

image

Map

image

The solution is deployed to the BizTalk runtime and configured through the BizTalk Administration Console. The interesting part is the configuration of the receive location using the SB-messaging adapter.

Configuring the SB-Messaging Adapter

image

The first tab for configuring the SB-Messaging is setting the URL for Queue (see picture of the Service Bus Queue: QUEUE URL). In the Second Tab the Authentication you provide the management credentials of the namespace (BTUGNLD).image

The final tab properties you can enable promotion of Brokered Message Properties.

image

The configuration of the receive location is straight forward as you can see. The information on configuring this adapter in a receive location can be found on MSDN: How to Configure an SB-Messaging Receive Location.

Running the solution

When I run the client application a form will appear, where I can fill in the details of a runner.  image
After filling in the detail I hit submit. When a message is send I can see the duration of sending the message from the client to Service Bus Message Queue. On average it takes less than a second. On my VM in Windows Azure I query the RegisteredTimes table and will see the details of the runner.  image

Latency

Sending one message to Service Bus queue takes less than a second. Sending a 100 messages (setting participants to 100) will take about 2 seconds.

Number of Messages Duration (Latency) in seconds Messages/Sec
1 0.665 1.50
10 1.59 6.29
100 9.15 10.92
200 14.39 13.89
500 35.84 13.95
1000 68.07 14.69
10000 707 14.14
100000 ~7142 ~14

As soon as the client is finished with sending the messages I instantly check the database in my VM and query number of records in the table. All the records are there. So it's safe to say that latency to have all messages pulled from the queue and inserted in SQL table is around the same time (i.e. duration to send to queue and few milliseconds later). With number of messages increasing the average duration (latency) is far less than a second to send a message to queue and then pick it up from queue to insert the message (data) into a table.  Throughput

Now 100000 messages would take about an 2 hours (extrapolate with measured data below) to be inserted in SQL Server table.

image

Usage Overview

Below you can see the number of messages going through the Service Bus Queue “registeredtimes”. There is a latency of a few minutes before actual number are shown (it is near real time).  image

Overall I am very impressed with this solution. Messages sent so quickly to a Service Bus queue and are then picked up by BizTalk Server 2010 R2 using the SB-Messaging adapter to store data in SQL Server 2008 R2.  Image that I would like to register all the finishing times of a marathon with 50000 participants or all finishing times including passing the 5, 10, 15, 20, 25 ,30, 35 and 40 km mark. That would accumulate in 450.000 messages. This can be quite some time (a few hours, see throughput) so scaling out with more queues, tune SQL Server and BizTalk would be a way to shorten the time.

You could of course use a queue for 5, 10, 15, 20, 25 ,30, 35, 40 km and finish mark. Or you could use topics and subscriptions for this. In my next post I will discuss a solution that will support this scenario using queues and topics/subscriptions including some fine tuning of the BizTalk / SQL Server. Interesting to see the latency and throughput then.

Stay tuned!