This post was originally published here
Azure Enterprise Integration Series
Part 1: Challenges in Enterprise Integration Development in Azure
1. Why the need for this series
It’s been nearly 9 months now since Logic Apps went GA, and there’s starting to be real growth in adoption and usage. We’re also seeing a lot more community content, as developers and companies start to see the real benefits of cloud integration and, specifically, the use of Logic Apps.
But for all this, there is a lot that is missing. If you’re new to Logic Apps, especially if you’ve come from a more formal development background (e.g. BizTalk or C# development) you’ll notice that there’s not a massive amount of development or pattern guidance out there.
2. What is Enterprise Integration Development?
For our purposes, we’ve defined Enterprise Integration Development as development that:
- Follows a known and proven process
- Scales to hundreds of developers and thousands of interfaces
- Uses DevOps where possible
- Uses integration patterns
- Produces quality code
- Thinks ahead about production and maintenance
- Provides guidance on operations and monitoring
3. BizTalk Development
Let’s assume you’re a BizTalk developer: You’re used to how BizTalk development works.
For example, some of the tasks you perform may include:
- Receiving a design brief or requirements
- Preparing your development machine (which will likely have BizTalk, Visual Studio, SQL Server etc. on it)
- Following guidance on naming conventions, logging, exception handling, and common patterns
- Creating a BizTalk solution and projects on your development machine
- Creating BizTalk artefacts (pipelines, schemas, transforms, orchestrations, etc.)
- Probably creating at least one C# library project (with either helper classes, or custom components)
- Creating unit tests
- Creating integration tests
- Deploying your code to your development machine and testing it
- Monitoring your BizTalk system and viewing log files
- Checking in your code to a source repository (e.g. VSTS)
- Deploying your code to other environments using either BTS console, or Deployment Framework, or scripts in VSTS
And if you don’t know how to do any of the above, you’ll be able to easily find out online, or look in one of the many BizTalk books.
4. Azure Integration Development
Now compare this to Azure Integration Development. If you’re new to Logic Apps, you’ll find that it’s not clear what the equivalents to the above steps are: Cloud development in Azure works differently.
For example:
- There’s no concept of deploying your code to local development machine – instead you must deploy to a subscription in Azure
- There’s limited or no guidance on naming conventions, logging, exception handling or common patterns
- Do you do development in the Portal, or Visual Studio, or both?
- There’s (currently) no book to refer to
- It’s unclear how you create some resources in Visual Studio (e.g. how do you create a Service Bus namespace? Or a Storage Account?)
- Little guidance on unit testing or integration testing of resources
- It’s not always clear how resource should be deployed e.g. how do you deploy an Integration account from Visual Studio?
- What do different environments (e.g. Dev, test, UAT, Prod) look like in Azure? Are they different subscriptions? All in one?
- What extra considerations do you need to think about that you may not be aware of?
Azure integration development also introduces a series of unique challenges that we didn’t have with local development.
For example:
- The Logic Apps designer will discover other Logic Apps, API Apps, and Functions… but only if they’re deployed at the time you are creating your Logic App: the designer can’t find resources that exist in Visual Studio buy which haven’t yet been deployed
- The Logic Apps designer can (currently) only discover resources which are in the same region and subscription
- Every time you execute a Logic App, you’re charged some money (assuming you’re on a consumption plan and have no free credits). If your developers have MSDN subscriptions, then they each get USD$50-$150 of free credits to use in their own subscription – but then they can’t see code in other subscriptions (see above)
- When you design an Integration Solution, you should think about the runtime cost, and calculate what the yearly/monthly costs might be, and factor this into your design
- Some resources (e.g. Integration Accounts, premium API Connections) incur extra cost, so use of them should be shared where possible
5. How we’ll address these questions and issues
This series of videos and blog posts aims to answer most of those questions and address these issues, by providing guidance on how to develop integration solutions in Azure. It’s based on our experience, at Affinus, on working with Logic Apps and Azure integration for over a year, both internally and for our clients.
If you’ve been using BizTalk for a while, you might remember that when BizTalk 2004 first came out, there was very little guidance or patterns or conventions. But then the BizTalk 2004 Unleashed book came out, and the Bloggers Guide to BizTalk was released, and then various forums and blog posts appeared, and developers gained experience and gradually a network of community content and support appeared.
The same will happen with integration development in Azure: we’re at the early exciting stages of integration PaaS development, and what’s more, Microsoft have moved to a new agile model where new features and capabilities are being released on a 2-weekly cycle.
Whether you’re a single developer, a small-medium business, or a large enterprise, Azure integration has a lot to offer.
6. The solution we’ll build
This series is all about practical examples and samples, hence the video webcasts.
In this series, we’ll be creating a moderately complex integration application that will consist of the following:
- Web App
- API App
- Function App
- Logic Apps
- Integration Account
- Service Bus
- Connectors: SalesForce, SQL, Service Bus