INTEGRATE 2018 – Recap of Day 2

This post was originally published here

Missed the Day 1 at INTEGRATE 2018? Here’s the recap of Day 1 events.

0830 — An early start on Day 2. The session started with consideration of using Logic Apps for System to System, Application to Application integration. 

Logic Apps finds it applications in multi-billion-dollar transactions that happen through Enterprise Application Integration platform. 

Most of these business cases are in BizTalk Server. Using Logic Apps and other Azure Services can modernise these platforms. 

Microsoft Vision

We do integration at the speed of the business. We want to simplify the process 

Microsoft is building tooling to automatically onboard partners and enable migration from BizTalk Server to Logic Apps. Embracing the change from taking the lead.

They are looking at reducing the DevOps time from Code -> Development -> Production

Microsoft is willing to share what they build to partners and community. Lot of integration patterns being built.

Microsoft is working on strategy to migrate from BizTalk Server to Logic Apps. How they plan to do?

• Use On-Premise Gateway
• Publish / Subscribe Model (work-in-progress)

The idea is to use integration workflows, publish to queues and there will be a subscriber reading from Queues to SAP / SFTP depending on the location.

They also mentioned about the categorization of Logic Apps under the following verticals:

• Policy / Route
• Processing
• LoB (Line of Business) Adapters

Suggestion is Have Logic Apps simple and self contained.

As more protocols are being added, they will update the policies. Also help decouple platform from onboarding.

• APIM Policies make it simple to drive itinerary
• Policies allow for dynamic routing of messages
• What properties need to be promoted can be derived from meta data

This means same logic app can be used for different partners and purposes by using routing and metadata

Exception Handling

• Enumerate over failures in run scope, extract properties relevant to business
• Forward to another logic app

They are suitable in Warm Path and Hot Path Monitoring.

Future Considerations

Integration Account to be used for

• BizTalk Server Partner configurations
• Logic Apps for Partners

Microsoft is ready with Migration strategy for the same

Logic Apps are being built suitable for isolated businesses provide offering to meet strongest SLAs

Amit demonstrated the following

1. TPM Management tool

• Help self-service onboarding
• Accelerate EDI integration

2. AB Testing tool

Basically, Microsoft is aimed at Simplification and Acceleration of Migration and on boarding process

Finally, they concluded the session with the learning to migrate to Logic Apps

0915 — API Management deep dive

After the first session of day 2 Vladimir Vinogradsky took the stage to talk about API Management which is one of first Azure service that developers gets introduced too.

Vladimir has started of with list of features that Azure API Management offers and how it helps developers to consume, mediate and publish their web API’s. In this he has explained how you can open the azure portal and look at the documentation of your APIs which is powered by Swagger without writing single line of code.

He also went in to detail on authentication and authorization and explained how Azure API management will help you with various authentication mechanisms from simple username password combo to Azure AD authentication etc. He also mentioned how easy it is to use third party providers such as Google, Facebook with Azure API management.

Vladimir then went into detail on what are all the frequent questions from API developers and how Azure API management can solve that. He explained this using some live demos which was well received from the audience.

1000 — Logic Apps Deep Dive

Kevin started with explaining Task Resiliency in Logic Apps

The highlight of the session was the demonstration of Integrated Service Environments (ISE) and its Architecture – but this is in the pre- private preview, means we need to wait for quite sometime.

Private Static IPs for Logic Apps are released with ISE

The deployment model of Logic Apps was also discussed with

Base Unit:

• 50 M action executions / month
• 1 standard integration account
• 1 enterprise connector (includes unlimited connections)
• VNET connectivity

Each additional processing unit

• Additional 50M executions / month

Logic Apps now has more than 200 Custom Connectors built. The Component Architecture of Logic Apps goes was also discussed.

1115 — Logic Apps Patterns & Best Practices

Kevin started this session with introducing the following patterns:

Work Flow patterns –

• Patterns are derived to implement Error Handling at work flows
• Define Retry policy – turn on/off retries, custom retries at custom and fixed rate as required for business.
• Run After patterns help in running logic apps after failure or time out. Limit can also be set for eg. You can stop Logic App execution after 30 seconds
• Patterns for Termination of execution of Logic Apps and associated Run Actions
• Scopes will have final status of all actions in that scope
• Implementation of Try-Catch-Finally in Logic Apps
• Concurrency Control for

1. Runs
   1. Instances are created concurrently
   2. Singleton trigger executions include level parallelism
   3. Degrees of parallel execution can be defined
2. Parallel Actions
   1. Explicit Parallelizations
   2. Join with Run After patterns
3. For Each Loops
4. Do Until Loops

• Patterns are discussed for Scheduling Executions. Example workloads can be clean up jobs
• Logic Apps can execute Run Once jobs. Example workloads can be time based jobs i.e. when you want to fire the action

Logic Apps support Messaging Protocols like:

• REST/SOAP
• Workflow Invocation
• Queues
• Pub/Sub
• Event Streams
• Eventing

These provide workflow invocation and componentization of Logic Apps

Messaging Patterns

Kevin then discussed the patterns for messaging that are categorized as

• Messaging Communication Patterns
• Messaging Handling Patterns

Derek Li provided some Best Practices

• Working with Variables
  • Variables in Logic Apps are global in scope
  • Array is heterogenous
  • Care needs to be taken when using variables in parallel for-each loop
  • Sequential for-each comes handy for having order

Derek Li made an impressive demo on how to efficiently use collections and parallel executions to process messages working with Arrays.

He also made a comparison of executing an array in Logic Apps in different possibilities that provided interesting inference.

He also made a comparison of executing an array in Logic Apps in different possibilities that provided interesting inference.

1200 — Microsoft Integration Roadmap

The last session before the lunch break was presented by Jon Fancey and Matt Farmer. The presentation was short and to the point. The audience got a view of the past and Microsoft plans towards future of integration.

Initially, we got a quick glimpse of all things that have been released as part of pro-integration in the past year. To emphasise that Microsoft is doing hard work in the integration space Jon and Matt announced that Microsoft has been recognised as a leader of enterprise integration in 2018 by Gartner.   

Next came the interesting stuff, what Microsoft has in plans for the Logic Apps?

1. Smart Designer – as seen in the other demos from Jeff and Derek they want to make the designer more user-friendly. They are looking into getting improved hints, suggestions and recommendations that actually apply to what you are using inside Azure.
2. Dedicated and connected – for all the companies that care about the security of their integrations, Logic Apps will be available in a vnet.
3. Obfuscation – another feature that will make the Logic Apps more secure within your organisation. Obfuscation will allow you to specify certain users that will be available to see the output of Logic App run.
4. On-Prem – Logic Apps are coming to Azure stack.
5. More: OAuth request trigger, China Cloud, Manage serviced identity, Testability, Manage serviced identity, Key Vault and Custom domain name for Logic Apps

Lastly, Jon and Matt revealed that they want to club every key azure integration such as Logic Apps, Event Grid, Azure Functions under one umbrella called Azure Integration Services.

The aim is to create a one-stop platform that will supply all the tools needed to fulfil your requirements in order to effortlessly bring your integrations to production in minimal time and maximum results. A platform that will allow you to run your integrations wherever you need it and however you need it, serverless or on-prem. We were told that Microsoft will provide guidance and templates across all the regions.

They finalised by acknowledging that there is still a lot to do and that many systems are not yet possible to connect with, but they strive to get everything into the platform including BizTalk, which they still see as part of the integration picture.

1330 — Post lunch, Duncan Barker, Business Development Manager at BizTalk360 thanked all the partners for their continued support. Then Saravana took stage to demonstrate the capabilities  of BizTalk360 and ServiceBus360.

Highlight of this session — ServiceBus360 will be re-branded into Serverless360. For more updates, please read this blog post.

1415 – Serverless Messaging with Microsoft Azure

Steef started with introducing the concept of Severlesss with the evolution from VM -> Containers -> IaaS -> PaaS -> Serverless.

Serverless reduces Time to Market, billed at micro level unit, reduced DevOps.

Messaging in Serverless is like Down, Stay, Come i.e you retrieve message when you want and got good control over message processing

The categorization of Serverless in Azure looks like:

There are various applications for Messaging in Serverless

• Financial Services
• Order Processing
• Logging / Telemetry
• Connected Devices
• Notifications / Event Driven Systems

Azure Serverless Components that support messaging are – ServiceBus, EventHub, EventGrid & Storage Queues

Steef provided lot of demo scenario for Messaging applications like

• Serverless Home Automation (Used Queues & Logic Apps)
• Connecting to Kafka Endpoint
• Toll Booth License plate recognition, that included IoT, OCR and Serverless components. He also used functions to process images.
• Pipes and Filters cloud patterns
• Microservice Processing
• Data and Event PipelineHe suggested some messaging considerations:
• Protocol
• Format
• Size
• Security
• Frequency
• Reliability
• Monitoring
• Networking

1450 — What’s there & what’s coming in Atomic Scope

After the session on Serverless Messaging with Microsoft Azure, Saravana took the stage to present on Atomic Scope, our brand new product from Kovai Limited. Even though there was an considerable amount of interest in Atomic Scope throughout the day 1 of integration event, most of the participants haven’t got the chance to fully experience the product due to various reasons like time constraints etc.

Saravana started explaining the challenges of the end to end monitoring when it comes to BizTalk and Hybrid integration scenarios and provided a couple example business process belonging to different domains. He also then proceeded to explain list of things that Atomic Scope tries to address like security end to end business visibility etc.

He then pointed out how much amount of effort that Atomic Scope can reduce when compared to the typical custom implementation solution for the end to end monitoring. Then Saravana went little bit deeper and explained how Atomic Scope actually works.

With that Saravana handed over the session to Bart from Integration Team and he showed how solution like BAM will not be sufficient for end to end monitoring and then he proceeded to explain a production ready atomic scope implementation which can add better value than BAM. Bart showed step by step process of how you can configure the business process and how tracking has happened once he dropped an EDI message to the APIM and On premise.

Bart’s presentation on Atomic Scope was very well received so much so that the AtomicScope booth was bombarded by participants after the presentation with people showing interest in the product and from people who was just curious to know more about the product.

1600 — BizTalk Server: Lessons from the Road

Sandro being a great lover of biz talk spoke about best practices such as making use of patterns , naming conventions , logging and tracing etc .

1640 — Using BizTalk Server as your Foundation to the Clouds

The last session of the day was done by Stephen M. Thomas, who gave his view on how BizTalk Server can be used as a foundation to using the cloud. Before he began his actual session, he introduced himself and mentioned a few resources he has been working on for learning purposes of Logic Apps. Amongst them are few Pluralsight trainings and hand-outs which can be found at his web site (http://www.stephenwthomas.com/labs).

The session consisted of two parts, being the Why you could use BizTalk Server combined with Logic Apps and Friction Factors which could prevent you from using them.

Why use BizTalk Server and Logic Apps

Stephen admits that Logic Apps don’t fit all scenarios. For example, in case your integrations are 100% on-premise or you have a low latency scenario at had, you could decide to stick with BizTalk Server. However, the following could be the reasons to start using Logic Apps:

• use connectors which do not exist in BizTalk Server
• load reduction on the BizTalk servers
• plan for the future
• save on hardware/software costs
• want to become an integration demo

Stephen sketched a scenario in which the need of connector exist which are not available in BizTalk. Think of for example scenarios for:

• social media monitoring, in which you need Twitter, Facebook or LinkedIn connectors
• cross team communication, in which you need a Skype connector
• incident management, in case you would need a ServiceNow adapter

Before Stephen showed few demos on batching and debatching with Logic Apps, he told that Logic Apps is quite good in batching, while BizTalk is not that good at it.

Friction factors

In the second part of his session, Stephen mentioned a number of factors which might prevent organisations to start using Logic Apps for your integration scenarios.

These factors included:

• We already have BizTalk (so we don’t need another integration platform)
• Our data is too sensitive to move it to the cloud
• The infrastructure manager says No
• Large learning curve for Logic Apps
• Azure changes too frequently
• CEO/CTO says NO to cloud

All these factors were addressed by Stephen, putting the door open to start using Logic Apps for your integrations, despite the given friction factors.w

With that, we wrapped up day 2 at INTEGRATE 2018 and it was time for the attendees to enjoy the INTEGRATE party over some drinks and music.

Read the Day 3 highlights here.

Thanks to the following people for helping me to collate this blog post—

• Arunkumar Kumaresan
• Umamaheswaran Manivannan
• Lex Hegt
• Srinivasa Mahendrakar
• Daniel Szweda
• Rochelle Saldanha