June 23, 2017
(This post was originally published on Mexia’s blog on 19th June 2017)
Microsoft recently announced that Azure BizTalk Services (MABS) is officially being retired. This was no great surprise, as those who actually used this service and its VETER pipelines to build integrations were well aware that the tooling was cumbersome, the DevOps story was terrible, scalability was severely limited, and the management capabilities left much to be desired. Logic Apps and the Enterprise Integration Pack already have already far surpassed the capabilities of MABS for cloud-based integration and B2B (EDI) scenarios. However, the one really useful feature of MABS was the free Hybrid Connections capability – free because this feature never made it out of preview mode.
Hybrid Connections allowed you to easily connect your Web App or Mobile service to an on-premises resource without making any changes to your corporate network, traversing NATS, routers, firewalls etc. with a purely codeless solution. In fact, you could literally “lift & shift” your existing on-prem website to Azure and not even have to alter the connection string to your database. Moreover, it worked at the transport layer so there was no dependency on WCF or .NET. I was so intrigued by the capabilities of this service that I authored a Pluralsight course on it, as well as creating a webcast and writing several blog posts.
With the obvious signs over the past year or so that MABS was on its way out, this had us wondering what would happen to Hybrid Connections? Other non-network related technologies like Service Bus Relay and the newer On-Premises Data Gateway certainly offer some viable alternatives, but nothing that permitted the same flexibility as Hybrid Connections. Fortunately, late last year we got our answer – the new Azure Relay.
A New Offering
Azure Relay became generally available on 27 March 2017, less than five months after the preview was announced. This service actually is comprised of two capabilities: the WCF Relay (which is the new name of the existing Service Bus Relay), and the new version of Hybrid Connections. This version of the latter is everything that the former version was, but much more:
- It is no longer hosted in a sunsetted technology (lives in Azure Service Bus)
- A published API means that the capability is no longer confined to Azure Web Apps and Mobile Services
- Reliance on web sockets means it is truly a cross-platform solution
In my Pluralsight course and in my previous webcast, I proved how easy it was to enable a single Azure hosted web site to talk to two separate on-premises resources (a web service and a SQL Server database). That capability exists in the new Hybrid Connections and can be set up in exactly the same way; a convenient downloadable manager agent can be installed in seconds which will complete the listener setup and allow you to flow messages into your network. I was easily able to recreate the same demo scenario in my webcast with the new version.
But even more compelling was the experience at using the API to build a more flexible solution, for example connecting an Azure hosted VM to an on-premises resource. Here, the supplied samples on GitHub (conveniently for both .NET and Node) really prove the extensive capabilities of this service.
The simple sample introduces you to the Hybrid Connections API by building a simple relay that bi-directionally exchanges blocks of text over a connection. I had this up & running in minutes just by deploying the server app on my local Hyper-V hosted VM and deploying the client application on an Azure-hosted VM. There were a few minor glitches to get over, for example how to correctly format the SQL Server connection string to talk to the local SQLEXPRESS instance I had running. This post helped me realise that I needed to explicitly include the port number in the connection string, as well as ensure that the TCP/IP settings for SQLEXPRESS used a static IP address (which is does not by default).
Of course you need to first create the Hybrid Connection in Azure, which you do in the portal by first creating an Azure Relay:
This sets up the Service Bus namespace that is associated with the relay artefacts. Then you can create the Hybrid Connection:
By going into the Shared Access Policies menu item for the Relay, you can copy the Primary Key for the “RootManageSharedAccessKey” – a necessary item for configuring the applications that will use the relay. (Of course you can and should configure your own policies for Manage, Send and Listen – but I didn’t for this example).
When you open any of the solutions from the GitHub samples, you will need to update the Microsoft.Azure.Relay Nuget package before compiling the solution:
Then it’s a matter of copying the solution to your servers, running the Client app on the Azure hosted machine and the Server app on the on-prem machine. I was impressed by the speed of the connection. Messages typed into the client appeared instantly in the server console window with absolutely no perceptible delay! Given the fact that under the covers, the relay uses Service Bus queues to transfer messages, I would have expected and happily tolerated a slight latency here – but there was none! I even opened multiple instances of the client to see how that would work:
As expected, all of the messages were visible in the server-hosted console, along with logging of sessions opening and closing:
Although the application in this format isn’t particularly useful for any production scenarios I can think of, it certainly proves that hybrid connectivity is no longer limited to Web Apps.
The next step was to try out the Port Bridge sample, which allows you to set up a hybrid connection that allows you to setup multiplexed socket tunnels that can bridge TCP and Named Pipe socket connections across the relay. This is an amazingly flexible demonstration as once you deploy the code, you can add SSH-like connections through configuration only, including imposing firewall-like restrictions – all without any involvement of your physical network configuration.
Again, I was keen to prove that this would allow connectivity across multiple nodes and ports, so I resorted again to my previous demo example and set up connections to both an locally hosted WCF service (on port 80) and my SQL Server database (on port 1433). The WCF service was really nothing more than a simple echo function that also outputs the name of the local server (same as used in my Guestbook demo in the webcast):
And for the database connectivity, all I needed to do was prove that a simple query would return the appropriate number of rows in a table:
I created two console test apps:
- One that called the WCF service
- One that executed a scalar query of the number of rows in the Refreshments table above
After testing both console apps locally on the Hyper-V VM, I then deployed these to my Azure VM, along with the portbridge solution (which was also deployed locally). The next step was to create the Hybrid Connection in Azure, which I did by adding to the existing Relay. The trick here was to ensure that the name of the relay (normally anything you like it to be) had to be the same name as your target host name (in this case, it was “devdan10”, the name of the VM hosted in my local Hyper-V).
Getting the configuration right took me a few goes, but eventually got there. Here is the configuration for the on-prem Server Agent console:
Notice on Line 10 that I’ve referenced the name of my target host (“devdan10”) and listed the port numbers which I will allow to be opened to the relay.
And here is the configuration for the Client Agent service running in Azure:
The local and remote ports happen to be the same here (lines 10 & 16), but they could be different – allowing you to map different ports between the remote client and the target local machine. The firewall rules in this case ensure that the requests come from the client machine – but these can also be adjusted to be as lenient or strict as desired.
There were no changes at all required for the WCF test client, in fact I was able to use “localhost” as the hostname in the client endpoint definition (line 53)!
One thing I did need to do was ensure that the connection string in the SQL test client used a TCP protocol and (again) a reference to the local machine rather than the target host name (this is presumably because the portbridge client service maps everything from the local machine on that port to the target remote server):
Note that you do need to use SQL Authentication rather than Windows Authentication, as there is no support for Active Directory authentication here.
With that in place I was able to run tests with both clients to prove connectivity over the ports I had configured for forwarding. The first test screenshot show the WCF client test app successfully invoking the service remotely. The window in the background is the console for the Port Bridge Client Agent that runs on the Azure machine, forwarding requests to the remote on-prem machine by way of the Hybrid Connection relay:
The Port Bridge Service Agent console windows runs on the on-prem machine, and logs the connections:
By the way, it is worth mentioning that both the Port Bridge Service Agent and the Client Agent can be installed and run as a Windows service.
The second test is the same thing but with the SQL Server database test client, correctly retrieving the row count from the on-prem machine:
How good is that?? You now have the capability to open of a port-based bridge between cloud and on-premises machines by adding configuration to the client and server agent services, meaning you can successfully connect your IaaS VMs and cloud services [almost] as easy as you connect your Web Apps and Mobile Apps.
In a week’s time I will be speaking about Hybrid Connectivity at the INTEGRATE 2017 conference in London. Unfortunately there won’t be time for demos, but this blog post might make for a handy follow-up reference.