The Global Integration Bootcamp 2017 – A successful event!

The Global Integration Bootcamp 2017 – A successful event!

Success has many fathers and with the Global Integration Bootcamp that is a fact. Organizers, speakers, companies hosting the event, the Microsoft Pro-Integration Team, and above all the attendees. This global event showed how passionate, strong and active this integration community is!

The Event

During the Global Integration Bootcamp at the Dutch location at motion10 in Rotterdam I did a few interviews with fellow organizers of this amazing event:

The motion10 has an amazing view over Rotterdam, which you can see in two of the video interviews.

In case you like to learn more about the history and how this great event came its tremedous success do read these two excellent posts:

Start of the day

In holland we kicked of at motion10 HQ at the same time as some of the other locations in Europe. Our friends in New Zealand, Australia and India just completed there day.

Sessions

At motion10 Gijs kick off the day followed by Tomasso presenting on API Management.

After Tomasso session the attendees dive into the first lab. The requirements to do the labs yourself can be found here and the labs will be published soon in an eBook by BizTalk360.

The second session was done by Rob Fox on Hybrid Integration including working with the on-premise gateway.

During the lunch people continued with their labs, followed by the third talk by Eldert, who continued with the hybrid integration story. This story included the connectivity through Logic Apps with on-premise BizTalk Server.

My session, number four, was to showcase some great Azure services like Logic Apps and Azure Functions. It was fun to do as I was able to show my trumpizer demo again.

The final session of the day was about IoT and done by Eldert again.

Acknowledgements

I am proud to be able to be part of this great event and journey to it. Loved the close collaboration with Eldert, who has been a key factor in organizing this event. His passion and drive is amazing and his recent MVP award is well deserved as it is a crown of his dedicated work for the community. And this something that I like to point out as his mentor. Well done buddy!

Thanks from the Dutch speakers #gib2017 and see you next year at #gib2018!

Cheers,

Steef-Jan

Author: Steef-Jan Wiggers

Steef-Jan Wiggers is all in on Microsoft Azure, Integration, and Data Science. He has over 15 years’ experience in a wide variety of scenarios such as custom .NET solution development, overseeing large enterprise integrations, building web services, managing projects, designing web services, experimenting with data, SQL Server database administration, and consulting. Steef-Jan loves challenges in the Microsoft playing field combining it with his domain knowledge in energy, utility, banking, insurance, health care, agriculture, (local) government, bio-sciences, retail, travel and logistics. He is very active in the community as a blogger, TechNet Wiki author, book author, and global public speaker. For these efforts, Microsoft has recognized him a Microsoft MVP for the past 6 years.

Global Integration Bootcamp 2017

Global Integration Bootcamp 2017

Yesterday, we held the first Global Integration Bootcamp, an event set up by integration communities around the world. We ended up doing the event in 12 countries, with 16 locations, over 55 speakers and an amazing number of over 650 attendees. Starting in New Zealand, and going around the globe, finally ending in Seattle, this truly has been a global event, bringing the community together.

I am proud to have been one of the organizers and drivers behind this great event, along with my good friends and community leaders, Steef-Jan, Rob, Tomasso, Martin, Gijs, Glenn and Sven.

Back in august, Mandi Ohlinger sent out the following tweet, about the BizTalk Bootcamp which had been done in Charlotte.

Mandi Ohlinger Following @mandi_onlinger Agenda is lookin' good. Great lineup of discussion leaders. aka.ms/BizTalkBootCamp #BizTalkBootCamp #LogicApps #msbts

The tweet quickly got picked up by the community, to do an event like this in more locations.

Gijs in 't veld @gintveld 22 Aug 2016 @mandi_onlinger now about a European counterpart? #msöts 1 Martin Abbott @martinabbott - 23 Aug 2016 @gintveld @mandi_onlinger and APAC tor that matter, f you need a body helping organise let me know 2 Glenn Colpaert @GlennColpaert - 23 Aug 2016 @martinaööott @gintveld @mandi_onlinger f you need a speaker, let me know (Can't blame a guy tor trying» 1 Glenn Colpaert @GlennColpaert - 23 Aug 2016 @martinaööott @gintveld @mandi_onlinger less liking, more inviting ; 1 Martin Abbott @martinabbott - 23 Aug 2016 @GlennColpaert @gintveld @mandi_onlinger leave t witn me.. I 1 Mandi Ohlinger @mandi_ohlinger - 23 Aug 2016 @martinaööott @GlennColpaert @gintveld Great idea guys. It needed, I can provide some guidance. 1 Eldert Grootenboer @egrootenboer - 23 Aug 2016 @mandi_onlinger @martinaööott @GlennColpaert @gintveld Would definitely be a great idea, let me know now I can help out' 1 Martin Abbott @martinabbott - 23 Aug 2016 @egrootenöoer @mandi_onlinger @GlennColpaert @gintveld I love the integration community #msöts

We quickly got a core group of people together, with whom we started moving this idea forward, and thus the organization of the Global Integration Bootcamp was formed. After a couple of Skype calls with Mandi and the group in the following weeks, we started to reach out to members of the community to see if we could expand our reach. It became clear that there definitely was a lot of interest for an event like this, starting with around 6 countries, so we started a Slack group, brought up a website, and opened registrations for locations. In the meantime, a lot of work has been going on behind the scenes. A couple of us started working on creating labs for the locations, thinking of a scenario and working this into fun labs, using the technologies Microsoft has given us over the years, like BizTalk, Logic Apps, Service Bus, API Management, On-Premise Data Gateway, IoT Hub and more!

As time progressed, more and more locations started contacting us to join in on the event as well, turning it into a real global event. We also worked closely with the MS Pro Integration product group during our preparations, where Jon arranged some very nice shirts and stickers for all the locations.

GLOBAL BO *ORLDWIOE

And we even got a video from the product group which the locations got to use in the welcome slides, with Jim welcoming everyone to the Global Integration Bootcamp, bringing the global communities even closer together.

Global Integration Bootcamp Welcome Video

And so finally, March 25th was here, and the Global Integration Bootcamp got on the way. As people started in New Zealand, twitter was starting to become busy on the #GIB2017 hashtag, and as the event started going around the globe, positive messages from around the globe started coming in, the Bootcamp was a success!

I got to do 2 sessions myself in Rotterdam, one being on Hybrid Integration using BizTalk, and the other on IoT with Azure, with Steef-Jan, Tomasso, Rob and Gijs doing sessions as well. We had a very nice turn-out, and people had a lot of fun watching the sessions and doing the labs.

C:UsersELDER_~1AppDataLocalTempmsohtmlclip12clip_image001.png

Below are some impression of the event, and reactions we had from the attendees, speakers and product group. Looking at the feedback we received, we decided we have to make this an annual event, so be prepared for Global Integration Bootcamp 2018! More information will follow in the coming weeks and months, keeping you up to date as we move forward. Thanks to everyone who helped us make this such a great event, my co-organizers, the locations organizers, the product groups, the attendees, and anyone I may have missed here! I will close this post with some impressions and feedback we got of the event. Be sure to check out the #GIB2017 hashtag on twitter for more, read the blogpost Saravana made about this as well, and join us at Integrate if you want to talk more about this years or next years Global Integration Bootcamp.

Tord Glad Nordahl Help Microsoft build Integration software know of very few products that have a community spinning up everything needed for a hands on m across multiple services and tools, book it on a Saturday cwer 17 cities, 12 countries and get over 700 people to sign up worldwide. Microsofts integration community is unique. And I am incredibly proud!

Jeff Hollan @jefhollan 20h Incredible to see #gib2017 around the world. @Microsoft integration stands apart because of our community. Lucky to consider myself a member

Jim Harrer @JimHarrer 7h Thank you for the support' Gijs in 't Veld @gintveld Microsott has spawned an awesome global #integration community @scottgu @JimHarrer Greetings trom the Dutch Team' #giö2017 @microsott #Azure

Tord Glad Nordahl @tordeman - 7h Hey, shoutout to the integration community, YOU ROCK! #msbts #gib2017

Jon Fancey @onfancey 10h more awesomeness from our fantastic community of MVPs and partners #gib2017 ROD FOX robfox The Rotterdam speakers ot #giö2017 @egrootenöoer @SteetJan @tlagroenendijk @gintveld @xroötox #mstt #azure #integration

J. Kunjur @jkunjur 10h #GlB2017 Chicago Global Integration 300tcamp pictures. VÉ had thirty attendees and positive teedöack_ Attendees came trom IN, Ml and WI.

Gyanendra Gautam @ggauta 10h dig thanks to all #GlB2017 organizers world wide' Happy to be part ot this great event'

Mattias Ldgdberg @MLogdberg 13h @TechMeet360 Great event' Great attendes and speakers. Alot ot tun and nard work _ Hope it was as good in India' Regards trom Sweden' #gib2017

Jet coo's @jcooIs85 16h Thanks to everyone around the globe who took partl organising the Global Integration 300tcamp' Top notch day ot integration tun' #GlB2017

Toon Vanhoutte @ToonVanhoutte - 17h dig thanks to all #GlB2017 organizers world wide' Happy to be part ot this great event'

C:UsersELDER_~1AppDataLocalTempmsohtmlclip12clip_image001.png

Global Integration Boot camp – a great success story

Global Integration Boot camp – a great success story

The Microsoft BizTalk Server community is one of the long-standing powerful technical communities in the world I’ve seen in an enterprise segment. For many years community members across the world have done numerous events like user group meetings, community gatherings, innovations days etc. This following LinkedIn statement by Tord Glad Nordahl who works as a Program Manager in BizTalk Server team at Microsoft is a testament to the above statement.

 

Tord Glad Nordahl

 

What is Global Integration Boot camp?

The idea of Global Integration Boot camp is pretty simple, uniting all the community members in the Microsoft Integration space across the world and conduct a one-day event on the same day.

Beginning of this year a group of people in the community along with Microsoft decided to go ahead with the plan. Hence the birth of Global Integration Boot camp (GIB).

The date was fixed to March 25th 2017 for the first GIB and a Slack group was set up for the core organizers to communicate. As you can see from the below screen shot from Slack there are 61 members in the group who worked behind the scene to make this event a great success.

Global Integration Bootcamp

Out of that 61 people, there are few outstanding contributors (core organising team) whom I wanted to personally acknowledge and thank for their relentless work behind the scene to make this event a grand success Eldert Grootenboer, Sven Van den brande, Rob Fox, Bill Chesnut, Martin Abbott, Dan Toomey, Glenn Colpaert, Lex Hegt, Steef-Jan Wiggers, Wagner Silveira, and all those people who contributed to GIB.

The format of the event is to have 3 to 4 sessions followed by a lab exercise in each location. The session and labs are focused on the following technology stack (all related to Integration)

BizTalk 2016 : BizTalk Server 2016, what’s new, and using the new Logic Apps adapter
Logic Apps : Creating Logic Apps using commonly-used connectors
Service Bus : Build reliable and scalable cloud messaging and hybrid integration solutions
Enterprise Integration Pack : Using the Enterprise Integration Pack (EIP) with Logic Apps
API Management : How does API management help you organize and secure your APIs
On-Premise Gateway : Connecting to on-prem resources using the On-Premise Gateway
Hybrid Integration : Hybrid integrations using BizTalk Server and Logic Apps
Microsoft Flow : Learn to compose flows with Microsoft Flow

Welcome message from Microsoft Product Group

The Microsoft Product Group from Seattle Jim Harrer, Jon Fancey, Tord Glad Nordahl, Kevin Lam, Jeff Hollan, Darek Li has put together this short intro video to GIB

What are the challenges?

Anything you do for the first time is going  to raise a great deal of scepticism whether something like this will work? The organisers did a great job of convincing people across different countries, the list started off some 5-6 countries initially and gradually expanded as the event got the moments. For example  a couple of locations were added just 2 or 3 weeks before the event date.

The second biggest challenge was securing venues at different places. In majority of the countries the event was conducted at Microsoft venues and in some places the partner organisations provided the space. It’s not just the space, for a whole day event like this there are a lot of logistics need to be addressed like food, drinks, and in some places security staff etc.

The organizers did an excellent job in addressing these challenges and moving forward seamlessly.

Locations, Organisers, Speakers and Sponsors

Following table summarises all the global locations in total 12 countries and 16 locations. Microsoft played an active role across all the locations in helping local sponsors, organisers to make this event a grand success.

CountryLocationOrganiserSpeakersSponsors
AustraliaBrisbane
  • Dan Toomey
  • Dan Toomey
  • Ashley Knowles
  • Lee Simpson
  • Mexia
AustraliaMelbourne
  • Bill Chesnut
  • Bill Chesnut
  • Nathan Fernandez
  • Sonja Bijou
  • Paco de la Cruz
  • Tanvir Chowdhury
  • SixPivot
  • InterprIT
  • Mexia
AustraliaSydney
  • Rene Brauwers
  • Rene Brauwers
  • Mick Badran
  • MOQdigital
BelgiumAntwerp
  • Sven Van den brande
  • Glenn Colpaert
  • Toon Vanhoutte
  • Steven Van Eycken
  • Glenn Colpaert
  • Jochen Toelen
  • BizTalk User Group
FinlandHelsinki
  • Sakari Nahr
  • Karl Ots
  • Karl Ots
  • Sami Ovaska
  • Teemu Tapanila
  • Kompozure
IndiaBengaluru
  • Arunkumar Kumaresan
  • Sriram Hariharan
  • Deepak Rajendran
  • Tulika Chaudharie
  • Harikharan Krishnaraju
  • Shailesh Agre
  • Shree Divya
  • Sunny Sharma
  • Lohith G Nagaraj
  • Sajith C P
  • Roy Joseph
  • Rekha Kodali
  • BizTalk360
United KingdomLondon
  • Saravana Kumar
  • Lex Hegt
  • Nino Crudele
  • Ashwin Prabhu
  • Michael Stephenson
  • Lex Hegt
  • Saffi Ali
  • BizTalk360
  • Codit UK
New ZealandAuckland
  • Wagner Silveira
  • Mark Brimble
  • Wagner Silveira
  • Nikolai Blackie
  • James Corbould
  • Morten Velling
  • Abhishek Kumar
  • Datacom
  • Adaptiv
  • Theta
NetherlandsRotterdam
  • Microsoft Integration User Group
  • Tomasso Groenendijk
  • Rob Fox
  • Eldert Grootenboer
  • Steef-Jan Wiggers
  • Motion10
NorwayOslo
  • Dominic Akadimah
  • Richard Hallgren
  • Communicate Norge
PortugalPorto
  • Sandro Pereira
  • Sandro Pereira
  • Ricardo Torre
  • José António Silva
  • Pedro Sousa
  • João Ferreira
  • Devscope
SpainMadrid
  • Kabel Sistemas
  • Mariano Robles
  • Carlos Sacristan
  • Felix Mondelo
  • Felipe Senso
  • Ivan Canizares
  • Francisco Nieto
  • Kabel Sistemas
SwedenStockholm
  • Johan Hedberg
  • Johan Hedberg
  • Mattias Lögdberg
  • Mikael Sand
  • Samuel Kastberg
  • Alan Smith
  • Mikael Håkansson
  • BizTalk User Group Sweden
United StatesChicago
  • Jaidev Kunjur
  • Jaidev Kunjur
  • Enkay Tech
United StatesNew York
  • Howard Edidin
  • Mandi Ohlinger
  • Howard Edidin
  • Stephen Thomas
  • Kent Weare
  • VNB Consulting
United StatesSeattle
  • Thomas E. Canter
  • Richard Seroter
  • Kevin Lam
  • Nick Hauenstein
  • Jeff Hollan
  • Phidiax
  • QuickLearn

Interesting Pictures from the event

These pictures shows the depth of the event across various locations 12 countries, 16 locations, 55 speakers and 650+ attendees.

Global Integration Bootcamp - NetherlandsGlobal Integration Bootcamp - Sweden
Global Integration Bootcamp - United KingdomGlobal Integration Bootcamp - Finland
Global Integration Bootcamp - SpainGlobal Integration Bootcamp - Belgium
Global Integration Bootcamp - PortugalGlobal Integration Bootcamp - India
Global Integration Bootcamp - NorwayGlobal Integration Bootcamp - Chicago
Global Integration Bootcamp - SeattleGlobal Integration Bootcamp - Australia - Melbourne
Global Integration Bootcamp - Australia - BrisbaneGlobal Integration Bootcamp - New Zealand
Global Integration Bootcamp - New YorkGlobal Integration Bootcamp - Australia - Sydney

Missed out on GIB? then don’t miss INTEGRATE 2017

If you have missed out all the fun of GIB 2017, then don’t miss out attending INTEGRATE 2017. It’s bigger in scale, all the Microsoft Product Group team and majority of the speakers who participated at GIB 2017 will be present in a single location in London. The registrations are now open for INTEGRATE 2017 and filling super fast.

The post Global Integration Boot camp – a great success story appeared first on BizTalk360.

Adding Variables to Logic Apps

Adding Variables to Logic Apps

Update 2017-03-26: I’ve updated the article below to make it clearer that each separate Logic App execution instance has its own member variable instance, but that variables are shared (currently) between parallel iterations in a loop.

The latest Logic Apps refresh, on March 24th 2017, finally added support for variables.

Microsoft have taken an interesting approach with variables: unlike a BizTalk orchestration, where you define variables separately to your workflow, in Logic Apps they’re defined in new Variables actions. This means that they’re defined inline, similar to how you would define them in source code. There is also a new @variables(‘variableName’) expression that can be used to obtain the value of a named variable.

There are some limitations to variables though:

  1. They are currently only global in scope within a Logic App execution (same value is available everywhere within that Logic App instance)
  2. You can currently only initialize a variable, or increment its value – you can’t update the value in any other way
  3. Only Integer and Float values are supported

In this post, I’m going to look at how you can add variables to your Logic Apps; what you can use them for; and why there is such limited support for variables in this initial release.

How to add variables

You can initialize and increment variables using the two new Variables actions (you can find them by entering “Variable” into the action search box):

As mentioned above, there are currently only two options.

Selecting Initialize variable gives you an action that looks like this:

You can supply the variable name, give it a type (only Integer and Float supported), and give it an initial value.

Selecting Increment variable gives you an action that looks like this:

You can select an existing variable (numeric variable types only – Integer or Float – although those are the only types you can create at the mo!), and enter the amount you want to increment by.

Interestingly, you can put a negative value in here, and it will decrement by that much.

What you can do with variables

Numeric variables that can be incremented/decremented are useful for two main activities: maintaining the count in a loop; and selecting a specific value (by index) from an array.

Here’s an example to show you how you could use the current numeric variable support.

Let’s imagine we’re sent an array of cities, as a JSON message. However, we don’t want to process *all* the cities in the array: we just want to process a subset. The request includes start and end position properties that give us the subset of cities to process.

For example:

{

    “count”: 4,

    “startPos”: 1,

    “endPos”: 3,

    “cities”: [

        {

            “city”: {

                “name”“New York”,

                “lat”: 40.71,

                “long”: -74

            }

        },

(more cities removed)

    ]

}

In the example above, we’re going to be supplied 4 cities, but we only want to process cities 1-3.

Without variables, this is tricky to do in Logic Apps, as we have no way of knowing where we are in a loop. Logic Apps only supports 2 types of loops currently: foreach, and do-until.

foreach iterates over all items in an array; do-until will iterate until a condition is reached. Neither of these options provide us a built-in way to iterate over a subset of items in an array.

Note: there are other ways to achieve this e.g. using the @take function, but none so clean as a for loop.

But now we can do this with Variables: by creating an Index variable, setting it to 0, using a foreach loop, and incrementing the index value. Unfortunately, there’s no equivalent of a for loop in LogicApps (i.e. where we can start at a given value and continue until a condition is met). So, that means we’re still going to iterate over every item in the array, but we’ll only process items that fall between the start and end positions.

There’s one other thing we need to do: we must set our foreach loop to execute sequentially i.e. not in parallel. By default, foreach loops will execute each iteration of the loop in parallel, up to 20 executions at a time. But trying to decrement a variable in a parallel loop gives us a non-deterministic value each time.

I learnt this the hard way, as I forgot to set my foreach loop to execute in sequence (in the example below), and I couldn’t work out why I was getting odd results…

Setting a foreach loop to execute in parallel is as simple as adding “operationOptions”: “Sequential” to the foreach definition. However, we currently need to do this from the code view as there’s no UI option to do this:

A For-Loop Example

Let’s create an example logic app that shows this.

We’ll take in the JSON message from above, which gives us a start position, and an end position, and then an array. And then we’ll initialize an Index variable to 0:

And then we’ll create a foreach loop. Because we don’t want to process all the items in the array, just those that are between the start/end positions, we’ll use a condition.

The condition we use needs to ensure that the current index is greater than or equal to the start position, and less than or equal to the end position. We’ll use the advanced editor and write it manually like this:

@and(greaterOrEquals(variables(‘index’), triggerBody()?[‘startPos’]),lessorequals(variables(‘index’), triggerBody()?[‘endPos’]))

Notice the use of the new variables expression.

If we are between start/end then we’ll send the city object to a service bus queue; if we’re outside the start/end position, then we do nothing. And then we’ll use the Increment variable action to increment the index variable by 1.

The for-each action looks like this:

We can test our Logic App by submitting a message to it using Postman, and if things go correctly, we should end up with 3 messages in our Service Bus queue:

Which is exactly what we get! (Note that I passed in the current Index variable value as the SessionId so I could debug.)

How about decrementing?

We can test that by initializing the index variable to the count of array items; and using “-1” as our increment value:

I tested using this message:

{

    “count”: 4,

    “startPos”: 2,

    “endPos”: 3,

    “cities”: [

        {

            “city”: {

                “name”“New York”,

                “lat”: 40.71,

                “long”: -74

            }

        },

        {

            “city”: {

                “name”“London”,

                “lat”: 120.82,

                “long”: -97.1

            }

        },

        {

            “city”: {

                “name”“Auckland”,

                “lat”: -127.45,

                “long”: 56.78

            }

        },

        {

            “city”: {

                “name”“Tokyo”,

                “lat”: -12.98,

                “long”: 34.66

            }

        }

    ]

}

I expected to get 2 more messages to my Service Bus queue, which is what I saw:

So, there we go, successful first test of using variables.

Why such limited support for variables?

One of my clients asked why we can’t yet set variable values i.e. what use are variables if you can only initialize them. And where’s support for string variables?

As the Logic Apps team mentioned in their last webcast, support for more variable types, and support setting a variable value, are coming.

But I can appreciate this is a tricky thing to do.

For example, let’s say you have a foreach loop, and you want to set a variable value on each iteration. How does that work if your loop is executing in parallel (the default)? Remember that the current variable action defines a variable that is effectively static within loop iterations i.e. all parallel runs that are spawned from a loop will have access to the same variable instance.

I suspect the product group will need to either find a way to scope variables (i.e. making changes to a variable in a particular execution in a loop doesn’t affect any other instances of that variable); or they may implement a version of the C# lock() keyword, so that only one thread can update the global variable at a time.

Note that this doesn’t mean that separate triggered executions of your Logic App share the same variable value: this only applies to parallel iterations in loops.

We’ll have to wait and see.

In any case, I hope this helps with your understanding of how to use variables in a Logic App. I’ll update this article, as more variable functionality is released.

Melbourne Global Integration Bootcamp–Live Webinar

Melbourne Global Integration Bootcamp–Live Webinar

For those that cannot make it to a Global Integration Bootcamp event live, we will be doing live webinar for the Melbourne, Australia event (recordings will also be available after the event)

Thanks to SixPivot for providing the facility to present the Live Webinars

You need to register for each session you would like to join using the link below.

Welcome & Session 1 : API Management & API Apps

https://attendee.gotowebinar.com/register/6751857195964514305

Join Bill Chesnut, Cloud Platform & API Evangelist for SixPivot, for the opening of the 2017 Global Integration Bootcamp in Melbourne, Australia and his session on API Management + API Apps

Session 2 : Service Bus, Enterprise Integration Pack & On-Premises Gateway

https://attendee.gotowebinar.com/register/7355755161427574787

Join Nathan Fernandez, Director for Interprit, for Service Bus + Enterprise Integration Pack + On-Premises Gateway

Session 3 : Logic Apps, Flow & BizTalk 2016 (Hybrid Integration)

https://attendee.gotowebinar.com/register/7705137075781361667

Join Tanvir Chowdhury, Principal Consultant for Mexia, for Logic Apps + Flow + BizTalk 2016 (Hybrid Integration)

Session 4 : Logic Apps (cloud adapters), Azure Functions & Azure Storage

https://attendee.gotowebinar.com/register/8961793104422407170

Join Paco de la Cruz, Senior Consultant for Mexia, for Logic Apps (cloud adapters) + Azure Functions + Azure Storage

Session 5 : IoT Hub, Stream Analytics, DocumentDB & PowerBI (IoT)

https://attendee.gotowebinar.com/register/9197003930371197954

Join Sonja Bijou, Solution Designer for Interprit, forIoT Hub + Stream Analytics + DocumentDB + PowerBI (IoT)

Logic Apps Update – March 2017

Logic Apps Update – March 2017

The Logic Apps team have been crazy busy this month. In case you missed the latest webcast (link here) this is what’s new and updated this month – and it’s a big list!!! 17 new features in all! Plus, one new region. And 17 new connectors. Here’s a look at all of that:

  1. Header Control
    For HTTP actions (e.g. HTTP or HTTP+Swagger, although not the Response action) you can now properly set headers in the designer – rather than using a JSON blob, you can enter name/value pairs directly in the designer – in the example below I’ve set the SOAPAction header:

    Note that the header value can also be set from a value returned by a previous step.

Also note that there’s a little “T” button at the top right of the Headers section – clicking this lets you switch back to the old JSON-based way of supplying headers.

 

  1. Generate schema from sample

The Request trigger gives you the option to specify a JSON schema for the message being received. If you do this, then you get rich UI support for the fields in your message i.e. they appear in the dynamic content boxes, so you can select the fields you wish to use).
To get the JSON Schema, you usually went to jsonschema.net to generate it. Now, you don’t have to – there’s a new option called “Use sample payload to generate schema” on the Request trigger:

When you click this, you get a window where you can paste in a sample JSON message:

When you do this and click Done, a new JSON schema is generated and added to the Request trigger:

Saves some time, although remember that if you’re working in Visual Studio, you’ll also want to save your JSON schema as a separate file, so it can be reused.

Note: JSONSchema.net gives you more advanced control over your schema, but for simple schemas this is a real time saver.

  1. Add a new action from anywhere in designer

This is a fairly subtle addition. Previously, if you wanted to add a new action to your logic app, you’d use the Add Action button that appears after the last action in a scope (e.g. last action in your logic app, or last action in a scope, or last action in a decision branch).

What wasn’t clear was how you add a new action in between two existing actions.

The way I always did it was to create a new action at the bottom, and then drag it up (did you know you can drag and drop actions?!) to the appropriate place. But apparently, some people did it from the code-behind.

Now, the team have got you covered:

  1. Table Action

This is a new action, that converts an array to html or csv. It basically allows you to pull out certain fields from an object array, and group those into an HTML template or a CSV stream.

It’s not yet available in the region I tested in, so I can’t see exactly how it works, but from the video you get two new actions, under Data Operations:

The demo showed using the Create HTML Table action:

You provide an array of objects, and it will return an HTML representation of those objects.

From the demo, it looks like it gives you an HTML table:

  1. Connection Resource Picker

This is a new feature that allows you to select the resource you want to connect to from within the designer. It currently supports ServiceBus, Document, and Blob storage. Previously, when you used the ServiceBus/DocDb/Storage connectors, you would have to enter a connection string for the resource you were accessing.

Now, the designer gives you a list of resources (that you can access via RBAC and which are in the same subscription), and you can pick the resource.

For example, the image below shows the use of the Service Bus GetMessage action, and it gives me a list of the Service Bus Namespaces I have access to – I can then click on one to select it:

Note: If you’re doing this from Visual Studio, you’ll have to first create the resources you want to access *before* you create your Logic App.

  1. Multipart formdata and formencoded

Some actions will return multipart/formdata or multipart/x-www-form-urlencoded data (or sometimes you just have an HTTP POST to a Logic App from a web form!).

Previously you’d have to go out a function or API App to parse this data. But now you can do this from within the Logic App, using several new expressions that parse the data and make it available easily.

  1. Terminate with Succeeded

The Terminate action is used to stop your Logic App at a certain point. Previously, the Terminate action could be used to indicate that your Logic App run had failed or been cancelled:

But now there is an option for your Terminate action to show up as a successful run.

  1. Open Azure Function from designer

This option is exactly as it sounds: in the CallFunction action, there is a new context option to open the function directly:

Selecting this opens the Functions App designer in a new blade, meaning you can edit the function, save it, and return to the Logic App.

Note that this option will not work from Visual Studio, where you would edit your function via a Visual Studio project, but if you’re using DevTool-less environment, then this is a great timesaver.

  1. Azure Functions HTTP Trigger Support

Previously, you could only call a function from a Logic App if it used the Webhook binding (either C# or JavaScript/Node.JS).

But now you can also use Functions that that use the HTTP binding:

The function template for an HTTP Binding trigger is identical to that of a WebHook trigger; one of the main differences between the two is that you can access query string parameters in an HTTP trigger, as shown in the sample you get when you create an HTTP Binding function:

  1. Variables

Finally! Rock on!!! I’ve been waiting for this for years, it feels.

For those of you who use Logic Apps regularly, and come from the BizTalk world, you may have noticed that there was no variable support: there was no direct way to store a value in a Logic App, and then change it, or access it later.

There were ways around this (e.g. using metadata in an Integration Account, or storing your variable sin Blob Storage/Azure SQL) but these weren’t ideal.

As of today, we have variable support.

Initial variable support is limited.

What we get is:

  • Initialize a variable (e.g. loopCount = 0)
  • Increment a variable (e.g. loopCount = loopCount + 1)
  • Access a variable (using the @variable expression – @variable(‘loopCount’))

Variables are currently global, but it sounds like scoped variables (plus additional variable support) is coming.

There’s no documentation on how to initialize/increment variables yet, but it’s coming

  1. Multiple actions in Do-Until

This may seem like a no-brainer if you’re new to Logic Apps, but up until today, a Do-Until loop could only contain a single action. This mean that most developers would call a function or a child Logic App here to get past the limitation.

But now a Do-Until loop is its own scope, so you can add multiple actions into it:

Note: although the Do-Until supports multiple actions, when I tested it (today) there was no “Add Action” option inside the loop, instead I had to create new actions outside the loop, and drag them in.

  1. Nested Foreach

Again, this is an incremental change, but also super useful. Previously, you couldn’t put a foreach inside another foreach loop. Now you can.

Although this may seem surprising if you’re a sequential programmer, the reason this was hard to do lies with the way that Logic Apps handles loops: by default, the Logic Apps execution engine will parallelise each iteration of a foreach loop (up to 20 executions in parallel).

For example, if you have a loop that will run 10 times, the execution engine will spin up 10 separate instances of the actions in that loop, and they will all execute in parallel, and then serialise back after they’ve all finished.

You can disable this behaviour using the sequential value on the operationOptions property for the foreach action (see the Logic Apps workflow action and triggers guide).

I’m not yet sure what happens with parallelisation of nested foreach loops – do they all execute in parallel? I’d assume so, but there must a limit on the total number of parallel executions across all loops.

  1. Foreach support 100k items

Previously, you could only support arrays/collections with up to 5000 items using foreach. Now you can use arrays/collections with up to 100,000 items. Bear in mind that your array might be a JSON object array, with each object having a lot of data.

  1. Version Promotion

This is great news for operations staff. For a while now, Logic Apps has supported the idea of storing the previous versions of a Logic App every time you save it. And you could swap the latest version with the previous version. But now you can see a list of all the versions, and you can promote any of them to live.

This is great news for operations/support staff if they have to quickly roll-back a production Logic App to a previous version.

Having said that, you should be using a fully automated deployment process that automatically tests your logic Apps, in which case you’d never need this feature 😊.

You can access the list of versions from the Versions tab (under Development Tools) when you view the properties for  your Logic App:

Clicking on a version gives you the option to promote it:

  1. Date-time filter for runs

This is an extension to work done to help you filter/search the run history for a Logic App.

Up till now, you could only filter on the run status i.e. Failed, Succeeded, Cancelled:

But now, you’ll also be able to filter by Date/Time. However, this change doesn’t yet seem to have made it to the region I was using so I can’t test it. Here’s the (low quality) image from the webcast:

  1. Open Nested Logic App

This is a monitoring view change: when you’re looking at the run history for a Logic App, if your Logic App called a child Logic App, you couldn’t previously jump into the child Logic App definition – you had to go find the run history for that Logic App separately.

But now, it will jump into the monitoring view for that child Logic App’s run.

Again, this doesn’t seem to be enabled in my region, so I can’t test it.

  1. Partner Tile in OMS

This is an addition to the B2B support for Operations Management Suite (OMS). Now you can view information grouped by Partner, not just by protocol.

  1. New Region: Canada

The Canada Azure datacentres are now available for hosting Logic Apps. And then after that comes the UK!

17 new connectors in total:

  • Azure Data Lake
  • Azure Automation (calling runbooks from Logic Apps)
  • Azure Event Hubs (trigger and action)
  • Oracle DB (supports On-Premises Data Gateway)
  • Zendesk (helpdesk)
  • Bitbucket (Git hosting)
  • Eventbrite (event organisation)
  • Pipedrive (sales CRM)
  • Infusionsoft (small business CRM)
  • Basecamp2 (Project Management/Team Communication software)
  • Intercom (messaging system for sales/marketing etc.)
  • FreshBooks (small business accounting software)
  • LeanKit (support software for KanBan)
  • WebMerge (pdf/word/excel file generation online)
  • Pivotal Tracker (agile project management software)
  • Toodledoo (to-do lists and notes)
  • Muhimbi PDF (PDF converter)
  • Azure Blob trigger: can start a Logic App when a new blob entry is created in a container.
  • FTP metadata-only trigger: can trigger when a file arrives, but instead of getting the file, you just get the metadata, and you can do claim checks, or decide if you want to get the file.
  • SQL Stored Procedures on-premises: can now call stored procedures via the On-Premises Data Gateway.

They’re also working on some cool new stuff:

  1. Optical Character Recognition Connector

This is part of the Computer Vision API, and will run OCR over an image you provide, and then return the text for that image.

  1. Additional Variables support

Will give the option to update the value of the variable once the Logic App has started (now, you can only initialize and then increment a variable).

You’ll also be able decrement a variable value, and also append your variable to an array.

  1. Parallel Designer Support

This is something you can do today (e.g. using the splitOn expression), but you can only do it from code-view. Support for executing Logic Apps in parallel will be surfaced up into the UI.

  1. Function Parameters

Functions will have SWAGGER support, which means that the Logic Apps designer will be able to display function parameters in the dynamic code viewers, and you’ll be able to use those function parameters in other actions.

  1. Select

This will let you create new arrays from existing arrays by selecting items in the array that met certain conditions.

  1. Join

This lets you create a string from an array, by supplying a separator token – each item in the array is added to the string, separated by the separator token.

  1. Retry Monitoring

At the moment, the Logic Apps execution engine handles retries for you, but the information isn’t surfaced up to the monitoring view. This information will be surfaced up, so you can see when retries occur plus the results, time they took, etc.

  1. Expression Tracing

This is super cool: if you have complex nested expressions, and you’re debugging a run and want to see what the value of each of the expressions was, then you’re stuck – you can only currently see the final value. But this new feature will show you the value of each expression evaluation.

  1. Management Portal Updates

Instead of having to go to aka.ms/logicappsupdate, you’ll be able to get to the list of updates directly from the Management Portal.

  1. Export Logic App to Visual Studio

A long-awaited feature, you’ll be able to select a deployed Logic App (from Visual Studio) and import that Logic App into a Logic Apps project in Visual Studio. This will appear in the next version of the Logic Apps Tools for Visual Studio. I’m not sure if you’re exporting from Azure, or importing into Visual Studio… 😊.

  1. New Connectors in-progress

New connectors are on their way, including:

  • Service Bus Sessions (will enable convoy patterns, yay!!)
  • Azure Cognitive Vision (OCR)
  • SQL Trigger (trigger when new rows are created)
  • Oracle EBS (E-Business Suite)
  • ProsperWorks (CRM for Google products)
  • Benchmark Email (Email marketing services)
  • Inoreader (content and news reader)

Whew! And that’s it.

Don’t forget the Global Integration Bootcamp, coming to a location near you… https://www.globalintegrationbootcamp.com/

BizTalk Scheduled Task Adapter is now stable and officially available for BizTalk Server 2016

BizTalk Scheduled Task Adapter is now stable and officially available for BizTalk Server 2016

Almost 20 days since I release the beta version of BizTalk Scheduled Task Adapter and still no issues reported. I also have been testing intensely the adapter so it is fair to say that this version it is stable, so, I officially remove the “beta label” from this version. BizTalk Scheduled Task Adapter is finally officially available (version 6.0) and optimized for BizTalk Server 2016!

The BizTalk Scheduled Task Adapter is an in-process receive adapter that executes a prescribed task on a daily, weekly or monthly schedule. The adapter is configured entirely within BizTalk, all configurations are stored within the SSODB and can be exported and imported via binding files.

The schedule capabilities are similar to those available with the Windows Scheduled Task Service.

BizTalk Scheduled Task Adapter for BizTalk Server 2016

What is new?

New schedule capabilities:

  • You can now schedule for a message to be a trigger in a time period, for example, each hour, minutes or seconds, every day or on certain days (for example all days except weekends).

BizTalk Scheduled Task Adapter new schedule capability

What was improved or fixed?

Several reported issues were from previous BizTalk Scheduled Task Adapter versions were solved, for example:

  • In some scenarios, trigger message sent twice at the same time – Solved
  • Bug: Time change (DST) bug – Solved
  • Bug: BizTalk Scheduled Task Adapter SQL Timeout issue – Solved

You can download this new version of the Scheduled adapter in the adapter CodePlex project page:

Please fill free to try and use the adapter and If you find any problems or bugs please open issue on the adapter Codeplex site: https://biztalkscheduledtask.codeplex.com/workitem/list/basic and help evolve this community adapter You can also suggest new features, I am also open to new ideas.

You can also find the Installation process documented here: BizTalk Scheduled Task Adapter 6.0: Installation process

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc. He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.

Creating BizTalk Server 2016 Developer from Azure Gallery Image

Creating BizTalk Server 2016 Developer from Azure Gallery Image

Published by: Bill Chesnut

The process for creating a BizTalk 2016 Developer machine with 2016 has changed from the previous versions, the Azure Gallery Image does not have SQL, Visual Studio or BizTalk installed, so I will walk you through those steps here.

Start with either the Classic or ARM Azure Gallery Image (ARM Image show below)

image

Logon to your newly created machine

image

You might want to turn off the IE Enhanced Security Configuration or you will be prompted for every website you visit

image

The image does not contain Visual Studio or SQL so you will need to download those:

You might need to signup for a free Visual Studio Dev Essentials subscription on http://my.visualstudio.com

SQL 2016 Developer:

image

Visual Studio 2015 (Community is free, use Professional or Enterprise if you have an MSDN subscription)

image

SQL Server Management Tools – http://go.microsoft.com/fwlink/?LinkID=822301

You should now have these files in your download directory

image

Installing SQL 2016 Developer Edition

We are now going to install SQL 2016 Developer Edition on the image

Note: for this installation I will be using the Administrator (bizadmin) for all services, you can use the accounts dictated by your companies security policies

Installing SQL Management Tools

We are now going to install SQL 2016 Management Tools on the image

Double Click on the setup applicationimage
Click Installimage
Installation Complete, Click Closeimage

Installing Visual Studio 2015 Community Edition

We are now going to install Visual Studio 2015 Community Edition on the image

Note: if you have MSDN subscription, feel free to install professional or enterprise

Installing BizTalk Server 2016 Developer Edition

We are now going to install BizTalk Server 2016 Developer Edition on the image

Configuring BizTalk Server 2016 Developer Edition

We are now going to configure BizTalk Server 2016 Developer Edition on the image

Enter the “User name” and “Password” that you want to use for the BizTalk Server Services, I am using the administrators account but feel free to create and use another account

Click Configure

Note: Ignore the warning about administrator account, this is a developer machine not production

image
Click Nextimage
Configuration Complete

Click Finish

image

Installing LOB Adapters

We are now going to Install the LOB Adapters & SDK on the image

Double Click on Setupimage
Click “Install Microsoft BizTalk Adapters”image
Click Step 1… to install SDK

Follow Setup Wizard

Note: I typically do a Complete install in everything in the Adapter Pack on a developer machine

image
Click Step 2… to install Adapter Pack

Follow Setup Wizard

Note: I typically do a Complete install in everything in the Adapter Pack on a developer machine

image
Click Step 3… to install Adapter Pack (x64)

Follow Setup Wizard

Note: I typically do a Complete install in everything in the Adapter Pack on a developer machine

image
(Optional)
Click Step 4… to install Adapter for Enterprise Applications

Follow Setup Wizard

Note: I typically do a Complete install in everything in the Adapter Pack on a developer machine

image
Open the BizTalk Server Administration Console

Expand Tree, Right Click Adapters, Click New, Click Adapter

image
Type Name WCF-SQL, Select WCF-SQL from Adapter drop down list

Click OK

Note: this is the only adapter I typically add, but feel free to add the rest of the adapters if you development project is going to need them.

image

Installing & Configuring ESB Toolkit (Optional)

We are now going to Install and Configure ESB Toolkit on the image

Final Configuration for Developer Image

We are now going to do the final configuration on the image

Open Microsoft SQL Server Management Studio from the Menuimage
Connect to your database server

Click Connect

Note: full stop or period means local machine

image
Click “New Query”image
For a developer image of BizTalk is it not necessary to create transaction logs for the BizTalk Databases, so the command below will set all database into simple recovery mode so no transaction logs are created and not database backups are required

Execute the Query

select ‘alter database [‘+name+’] set recovery simple’ from master.sys.databases where database_id > 4 and state_desc = ‘online’

image
Copy the output of the previous command into a new query windowimage
Click Execute

Note: all the databases are in simple recovery mode so there is no need to configure the BizTalk Backup Job

image
To keep the tracking database for becoming large on the developer image, we need to setup the DTA Purge Job

Right click on the “DTA Purge and Archive (BizTalkDTADb)”

Click Properties

image
Change the name to “DTA Purge (BizTalkDTADb)”

Click Steps

image
Edit the Stepimage
Change the step name to “Purge”

Update the Command to the text below

Click Ok, Click Ok

declare @dtLastBackup datetime
set @dtLastBackup = GetUTCDate()
exec dtasp_PurgeTrackingDatabase 1, 10, 11, @dtLastBackup

image
Right click on the “DTA Purge (BizTalkDTADb)”

Click Enable

Note: you can then do the same and select “Start Job at Step” to make sure you don’t have any syntax errors.

image

Conclusion

This finished the basic configuration with a few extra, if you are using BizUnit, BTDF, SpecFlow or any other tools, you will need to install them now.

I would also recommend running windows update to get the latest updates installed

I hope this instruction and screen shots help you get your BizTalk Server 2016 Developer Azure Gallery Image up and running smoothly

Logic Apps review (Usability and Extensibility) part 2

Logic Apps review (Usability and Extensibility) part 2

My previous post here.

https://blog.ninocrudele.com/2017/03/10/logic-apps-review-ui-and-mediation-part-1/

I keep going with my assessment with Logic Apps, I need to say that more I dig and more I like it and there are many reasons for that.
I love technology, I love it in all the aspects, one of the things that most fascinate myself is, what I like to call, the technology perception.I like to see how different people approach the same technology stack, Logic Apps is a great sample for me as BizTalk Server or Azure Event Hubs and other.
When I look to Logic Apps I see an open and fully extensible Azure stack to integrate and drive my azure processes.In these days I’m pushing the stack quite hard and I’m trying many different experiments, I have been impressed by the simplicity of the usability and the extensibility, I will speak more in detail about that in my next events but there are some relevant points of discussion.

Development comfort zone

Logic Apps offers many options, Web UI, Visual Studio, and scripting.
As I said in my previous post the Web UI is very fast and consistent but we can also use Visual Studio.
To develop using Visual Studio we just only need to install the latest Azure SDK, I’m not going to explain any detail about that as it is very to do and you can find anything you need here.
I just would like you to notice some interesting points, like the project distribution and the extensibility, first of all, to create a new Logic Apps process we need to select New Project/Cloud/Resource Group.

and we will receive a long list of templates, with the Logic Apps one as well.

I definitely like the idea of using the concept of multi templates as Azure resource group, I can create my own templates very easily and release them on GitHub for my team.
In Visual Studio we have two main options, using the designer just selecting the LogicApp.json file, right click and “Open using the Logic App designer” or directly using the JSON file.

The second by the editor which is able to offer the maximum granularity and transparency.

Deployment

About the development experience I need to say that it is very easy, just right click and Deploy

Extensibility

Logic Apps offers a lot of connectors, you don’t need to be a developer expert to create a new workflow, it is very simple to integrate an external stack and implementing a workflow able to exchange data between two systems, but my interesting has been focused on another aspect, how much simple is to extend and create my own applications blocks, triggers and actions.
I had a look at the possibility to extend the triggers and actions and I need to say that is very simple, we have two options, we can navigate in GitHub and start using one of the templates here, Jeff Hollan has created an impressive number of them and all of them are open source, I normally use these repositories to get the information I need or we can just create a new one from scratch.
Well, how much easy to create a new one from scratch, from Visual Studio we need to select New Project/Aps.Net Web Application/Azure Web API, set our Azure hosting settings

and we only need to implement our Get and GenerateAsyncResponse method able to provide the HttpResponseMessage that the Logic App flow needs to invocate for the trigger.

Continuous Integration

For the deployment, we can publish from Visual Studio or, very interesting and option for continuous integration, using GitHub, I tried both and I have been very satisfied by the simplicity of using GitHub.
To integrate GitHub we just need to create a new deployment slot in our Web API

The slot is another Web API by REST able to integrate the source control we like to use, we need to select our new slot

and select the source type we like to use, absolutely easy and fast, great job.

Telemetry

I also appreciated the possibility to use the Live Data telemetry which is very useful during the tests and the development and we have the same thing in the Azure portal.

I have so much to say about Logic Apps, you can find a lot of material and resources on an internet and please feel free to contact me for any more information.
I’m also experimenting many interesting things using Logic Apps in conjunction with BizTalk Server, I will speak a lot more in detail my next events, I definitely getting so much fun playing with it.

Author: Nino Crudele

Nino has a deep knowledge and experience delivering world-class integration solutions using all Microsoft Azure stacks, Microsoft BizTalk Server and he has delivered world class Integration solutions using and integrating many different technologies as AS2, EDI, RosettaNet, HL7, RFID, SWIFT.

Microsoft Integration Weekly Update: March 20

Microsoft Integration Weekly Update: March 20

Do you feel difficult to keep up to date on all the frequent updates and announcements in the Microsoft Integration platform?

Integration weekly update can be your solution. It’s a weekly update on the topics related to Integration – enterprise integration, robust & scalable messaging capabilities and Citizen Integration capabilities empowered by Microsoft platform to deliver value to the business.

If you want to receive these updates weekly, then don’t forget to Subscribe!

On-Premise Integration:

Cloud and Hybrid Integration:

Feedback

Hope this would be helpful. Please feel free to let me know your feedback on the Integration weekly series.

Advertisements