Adding Variables to Logic Apps

by Daniel probert | Mar 25, 2017 | BizTalk Community Blogs via Syndication

Update 2017-03-26: I’ve updated the article below to make it clearer that each separate Logic App execution instance has its own member variable instance, but that variables are shared (currently) between parallel iterations in a loop.

The latest Logic Apps refresh, on March 24th 2017, finally added support for variables.

Microsoft have taken an interesting approach with variables: unlike a BizTalk orchestration, where you define variables separately to your workflow, in Logic Apps they’re defined in new Variables actions. This means that they’re defined inline, similar to how you would define them in source code. There is also a new @variables(‘variableName’) expression that can be used to obtain the value of a named variable.

There are some limitations to variables though:

1. They are currently only global in scope within a Logic App execution (same value is available everywhere within that Logic App instance)
2. You can currently only initialize a variable, or increment its value – you can’t update the value in any other way
3. Only Integer and Float values are supported

In this post, I’m going to look at how you can add variables to your Logic Apps; what you can use them for; and why there is such limited support for variables in this initial release.

How to add variables

You can initialize and increment variables using the two new Variables actions (you can find them by entering “Variable” into the action search box):

As mentioned above, there are currently only two options.

Selecting Initialize variable gives you an action that looks like this:

You can supply the variable name, give it a type (only Integer and Float supported), and give it an initial value.

Selecting Increment variable gives you an action that looks like this:

You can select an existing variable (numeric variable types only – Integer or Float – although those are the only types you can create at the mo!), and enter the amount you want to increment by.

Interestingly, you can put a negative value in here, and it will decrement by that much.

What you can do with variables

Numeric variables that can be incremented/decremented are useful for two main activities: maintaining the count in a loop; and selecting a specific value (by index) from an array.

Here’s an example to show you how you could use the current numeric variable support.

Let’s imagine we’re sent an array of cities, as a JSON message. However, we don’t want to process *all* the cities in the array: we just want to process a subset. The request includes start and end position properties that give us the subset of cities to process.

For example:

{

    “count”: 4,

    “startPos”: 1,

    “endPos”: 3,

    “cities”: [

        {

            “city”: {

                “name”: “New York”,

                “lat”: 40.71,

                “long”: -74

            }

        },

(more cities removed)

    ]

}

In the example above, we’re going to be supplied 4 cities, but we only want to process cities 1-3.

Without variables, this is tricky to do in Logic Apps, as we have no way of knowing where we are in a loop. Logic Apps only supports 2 types of loops currently: foreach, and do-until.

foreach iterates over all items in an array; do-until will iterate until a condition is reached. Neither of these options provide us a built-in way to iterate over a subset of items in an array.

Note: there are other ways to achieve this e.g. using the @take function, but none so clean as a for loop.

But now we can do this with Variables: by creating an Index variable, setting it to 0, using a foreach loop, and incrementing the index value. Unfortunately, there’s no equivalent of a for loop in LogicApps (i.e. where we can start at a given value and continue until a condition is met). So, that means we’re still going to iterate over every item in the array, but we’ll only process items that fall between the start and end positions.

There’s one other thing we need to do: we must set our foreach loop to execute sequentially i.e. not in parallel. By default, foreach loops will execute each iteration of the loop in parallel, up to 20 executions at a time. But trying to decrement a variable in a parallel loop gives us a non-deterministic value each time.

I learnt this the hard way, as I forgot to set my foreach loop to execute in sequence (in the example below), and I couldn’t work out why I was getting odd results…

Setting a foreach loop to execute in parallel is as simple as adding “operationOptions”: “Sequential” to the foreach definition. However, we currently need to do this from the code view as there’s no UI option to do this:

A For-Loop Example

Let’s create an example logic app that shows this.

We’ll take in the JSON message from above, which gives us a start position, and an end position, and then an array. And then we’ll initialize an Index variable to 0:

And then we’ll create a foreach loop. Because we don’t want to process all the items in the array, just those that are between the start/end positions, we’ll use a condition.

The condition we use needs to ensure that the current index is greater than or equal to the start position, and less than or equal to the end position. We’ll use the advanced editor and write it manually like this:

@and(greaterOrEquals(variables(‘index’), triggerBody()?[‘startPos’]),lessorequals(variables(‘index’), triggerBody()?[‘endPos’]))

Notice the use of the new variables expression.

If we are between start/end then we’ll send the city object to a service bus queue; if we’re outside the start/end position, then we do nothing. And then we’ll use the Increment variable action to increment the index variable by 1.

The for-each action looks like this:

We can test our Logic App by submitting a message to it using Postman, and if things go correctly, we should end up with 3 messages in our Service Bus queue:

Which is exactly what we get! (Note that I passed in the current Index variable value as the SessionId so I could debug.)

How about decrementing?

We can test that by initializing the index variable to the count of array items; and using “-1” as our increment value:

I tested using this message:

{

    “count”: 4,

    “startPos”: 2,

    “endPos”: 3,

    “cities”: [

        {

            “city”: {

                “name”: “New York”,

                “lat”: 40.71,

                “long”: -74

            }

        },

        {

            “city”: {

                “name”: “London”,

                “lat”: 120.82,

                “long”: -97.1

            }

        },

        {

            “city”: {

                “name”: “Auckland”,

                “lat”: -127.45,

                “long”: 56.78

            }

        },

        {

            “city”: {

                “name”: “Tokyo”,

                “lat”: -12.98,

                “long”: 34.66

            }

        }

    ]

}

I expected to get 2 more messages to my Service Bus queue, which is what I saw:

So, there we go, successful first test of using variables.

Why such limited support for variables?

One of my clients asked why we can’t yet set variable values i.e. what use are variables if you can only initialize them. And where’s support for string variables?

As the Logic Apps team mentioned in their last webcast, support for more variable types, and support setting a variable value, are coming.

But I can appreciate this is a tricky thing to do.

For example, let’s say you have a foreach loop, and you want to set a variable value on each iteration. How does that work if your loop is executing in parallel (the default)? Remember that the current variable action defines a variable that is effectively static within loop iterations i.e. all parallel runs that are spawned from a loop will have access to the same variable instance.

I suspect the product group will need to either find a way to scope variables (i.e. making changes to a variable in a particular execution in a loop doesn’t affect any other instances of that variable); or they may implement a version of the C# lock() keyword, so that only one thread can update the global variable at a time.

Note that this doesn’t mean that separate triggered executions of your Logic App share the same variable value: this only applies to parallel iterations in loops.

We’ll have to wait and see.

In any case, I hope this helps with your understanding of how to use variables in a Logic App. I’ll update this article, as more variable functionality is released.

Logic Apps Update – March 2017

by Daniel probert | Mar 23, 2017 | BizTalk Community Blogs via Syndication

The Logic Apps team have been crazy busy this month. In case you missed the latest webcast (link here) this is what’s new and updated this month – and it’s a big list!!! 17 new features in all! Plus, one new region. And 17 new connectors. Here’s a look at all of that:

1. Header Control
   For HTTP actions (e.g. HTTP or HTTP+Swagger, although not the Response action) you can now properly set headers in the designer – rather than using a JSON blob, you can enter name/value pairs directly in the designer – in the example below I’ve set the SOAPAction header:
   
   Note that the header value can also be set from a value returned by a previous step.

Also note that there’s a little “T” button at the top right of the Headers section – clicking this lets you switch back to the old JSON-based way of supplying headers.

2. Generate schema from sample

The Request trigger gives you the option to specify a JSON schema for the message being received. If you do this, then you get rich UI support for the fields in your message i.e. they appear in the dynamic content boxes, so you can select the fields you wish to use).
To get the JSON Schema, you usually went to jsonschema.net to generate it. Now, you don’t have to – there’s a new option called “Use sample payload to generate schema” on the Request trigger:

When you click this, you get a window where you can paste in a sample JSON message:

When you do this and click Done, a new JSON schema is generated and added to the Request trigger:

Saves some time, although remember that if you’re working in Visual Studio, you’ll also want to save your JSON schema as a separate file, so it can be reused.

Note: JSONSchema.net gives you more advanced control over your schema, but for simple schemas this is a real time saver.

3. Add a new action from anywhere in designer

This is a fairly subtle addition. Previously, if you wanted to add a new action to your logic app, you’d use the Add Action button that appears after the last action in a scope (e.g. last action in your logic app, or last action in a scope, or last action in a decision branch).

What wasn’t clear was how you add a new action in between two existing actions.

The way I always did it was to create a new action at the bottom, and then drag it up (did you know you can drag and drop actions?!) to the appropriate place. But apparently, some people did it from the code-behind.

Now, the team have got you covered:

4. Table Action

This is a new action, that converts an array to html or csv. It basically allows you to pull out certain fields from an object array, and group those into an HTML template or a CSV stream.

It’s not yet available in the region I tested in, so I can’t see exactly how it works, but from the video you get two new actions, under Data Operations:

The demo showed using the Create HTML Table action:

You provide an array of objects, and it will return an HTML representation of those objects.

From the demo, it looks like it gives you an HTML table:

5. Connection Resource Picker

This is a new feature that allows you to select the resource you want to connect to from within the designer. It currently supports ServiceBus, Document, and Blob storage. Previously, when you used the ServiceBus/DocDb/Storage connectors, you would have to enter a connection string for the resource you were accessing.

Now, the designer gives you a list of resources (that you can access via RBAC and which are in the same subscription), and you can pick the resource.

For example, the image below shows the use of the Service Bus GetMessage action, and it gives me a list of the Service Bus Namespaces I have access to – I can then click on one to select it:

Note: If you’re doing this from Visual Studio, you’ll have to first create the resources you want to access *before* you create your Logic App.

6. Multipart formdata and formencoded

Some actions will return multipart/formdata or multipart/x-www-form-urlencoded data (or sometimes you just have an HTTP POST to a Logic App from a web form!).

Previously you’d have to go out a function or API App to parse this data. But now you can do this from within the Logic App, using several new expressions that parse the data and make it available easily.

7. Terminate with Succeeded

The Terminate action is used to stop your Logic App at a certain point. Previously, the Terminate action could be used to indicate that your Logic App run had failed or been cancelled:

But now there is an option for your Terminate action to show up as a successful run.

8. Open Azure Function from designer

This option is exactly as it sounds: in the CallFunction action, there is a new context option to open the function directly:

Selecting this opens the Functions App designer in a new blade, meaning you can edit the function, save it, and return to the Logic App.

Note that this option will not work from Visual Studio, where you would edit your function via a Visual Studio project, but if you’re using DevTool-less environment, then this is a great timesaver.

9. Azure Functions HTTP Trigger Support

Previously, you could only call a function from a Logic App if it used the Webhook binding (either C# or JavaScript/Node.JS).

But now you can also use Functions that that use the HTTP binding:

The function template for an HTTP Binding trigger is identical to that of a WebHook trigger; one of the main differences between the two is that you can access query string parameters in an HTTP trigger, as shown in the sample you get when you create an HTTP Binding function:

10. Variables

Finally! Rock on!!! I’ve been waiting for this for years, it feels.

For those of you who use Logic Apps regularly, and come from the BizTalk world, you may have noticed that there was no variable support: there was no direct way to store a value in a Logic App, and then change it, or access it later.

There were ways around this (e.g. using metadata in an Integration Account, or storing your variable sin Blob Storage/Azure SQL) but these weren’t ideal.

As of today, we have variable support.

Initial variable support is limited.

What we get is:

• Initialize a variable (e.g. loopCount = 0)
• Increment a variable (e.g. loopCount = loopCount + 1)
• Access a variable (using the @variable expression – @variable(‘loopCount’))

Variables are currently global, but it sounds like scoped variables (plus additional variable support) is coming.

There’s no documentation on how to initialize/increment variables yet, but it’s coming

10. Multiple actions in Do-Until

This may seem like a no-brainer if you’re new to Logic Apps, but up until today, a Do-Until loop could only contain a single action. This mean that most developers would call a function or a child Logic App here to get past the limitation.

But now a Do-Until loop is its own scope, so you can add multiple actions into it:

Note: although the Do-Until supports multiple actions, when I tested it (today) there was no “Add Action” option inside the loop, instead I had to create new actions outside the loop, and drag them in.

11. Nested Foreach

Again, this is an incremental change, but also super useful. Previously, you couldn’t put a foreach inside another foreach loop. Now you can.

Although this may seem surprising if you’re a sequential programmer, the reason this was hard to do lies with the way that Logic Apps handles loops: by default, the Logic Apps execution engine will parallelise each iteration of a foreach loop (up to 20 executions in parallel).

For example, if you have a loop that will run 10 times, the execution engine will spin up 10 separate instances of the actions in that loop, and they will all execute in parallel, and then serialise back after they’ve all finished.

You can disable this behaviour using the sequential value on the operationOptions property for the foreach action (see the Logic Apps workflow action and triggers guide).

I’m not yet sure what happens with parallelisation of nested foreach loops – do they all execute in parallel? I’d assume so, but there must a limit on the total number of parallel executions across all loops.

12. Foreach support 100k items

Previously, you could only support arrays/collections with up to 5000 items using foreach. Now you can use arrays/collections with up to 100,000 items. Bear in mind that your array might be a JSON object array, with each object having a lot of data.

13. Version Promotion

This is great news for operations staff. For a while now, Logic Apps has supported the idea of storing the previous versions of a Logic App every time you save it. And you could swap the latest version with the previous version. But now you can see a list of all the versions, and you can promote any of them to live.

This is great news for operations/support staff if they have to quickly roll-back a production Logic App to a previous version.

Having said that, you should be using a fully automated deployment process that automatically tests your logic Apps, in which case you’d never need this feature 😊.

You can access the list of versions from the Versions tab (under Development Tools) when you view the properties for  your Logic App:

Clicking on a version gives you the option to promote it:

14. Date-time filter for runs

This is an extension to work done to help you filter/search the run history for a Logic App.

Up till now, you could only filter on the run status i.e. Failed, Succeeded, Cancelled:

But now, you’ll also be able to filter by Date/Time. However, this change doesn’t yet seem to have made it to the region I was using so I can’t test it. Here’s the (low quality) image from the webcast:

15. Open Nested Logic App

This is a monitoring view change: when you’re looking at the run history for a Logic App, if your Logic App called a child Logic App, you couldn’t previously jump into the child Logic App definition – you had to go find the run history for that Logic App separately.

But now, it will jump into the monitoring view for that child Logic App’s run.

Again, this doesn’t seem to be enabled in my region, so I can’t test it.

16. Partner Tile in OMS

This is an addition to the B2B support for Operations Management Suite (OMS). Now you can view information grouped by Partner, not just by protocol.

17. New Region: Canada

The Canada Azure datacentres are now available for hosting Logic Apps. And then after that comes the UK!

17 new connectors in total:

• Azure Data Lake
• Azure Automation (calling runbooks from Logic Apps)
• Azure Event Hubs (trigger and action)
• Oracle DB (supports On-Premises Data Gateway)
• Zendesk (helpdesk)
• Bitbucket (Git hosting)
• Eventbrite (event organisation)
• Pipedrive (sales CRM)
• Infusionsoft (small business CRM)
• Basecamp2 (Project Management/Team Communication software)
• Intercom (messaging system for sales/marketing etc.)
• FreshBooks (small business accounting software)
• LeanKit (support software for KanBan)
• WebMerge (pdf/word/excel file generation online)
• Pivotal Tracker (agile project management software)
• Toodledoo (to-do lists and notes)
• Muhimbi PDF (PDF converter)

• Azure Blob trigger: can start a Logic App when a new blob entry is created in a container.
• FTP metadata-only trigger: can trigger when a file arrives, but instead of getting the file, you just get the metadata, and you can do claim checks, or decide if you want to get the file.
• SQL Stored Procedures on-premises: can now call stored procedures via the On-Premises Data Gateway.

They’re also working on some cool new stuff:

1. Optical Character Recognition Connector

This is part of the Computer Vision API, and will run OCR over an image you provide, and then return the text for that image.

2. Additional Variables support

Will give the option to update the value of the variable once the Logic App has started (now, you can only initialize and then increment a variable).

You’ll also be able decrement a variable value, and also append your variable to an array.

3. Parallel Designer Support

This is something you can do today (e.g. using the splitOn expression), but you can only do it from code-view. Support for executing Logic Apps in parallel will be surfaced up into the UI.

4. Function Parameters

Functions will have SWAGGER support, which means that the Logic Apps designer will be able to display function parameters in the dynamic code viewers, and you’ll be able to use those function parameters in other actions.

5. Select

This will let you create new arrays from existing arrays by selecting items in the array that met certain conditions.

6. Join

This lets you create a string from an array, by supplying a separator token – each item in the array is added to the string, separated by the separator token.

7. Retry Monitoring

At the moment, the Logic Apps execution engine handles retries for you, but the information isn’t surfaced up to the monitoring view. This information will be surfaced up, so you can see when retries occur plus the results, time they took, etc.

8. Expression Tracing

This is super cool: if you have complex nested expressions, and you’re debugging a run and want to see what the value of each of the expressions was, then you’re stuck – you can only currently see the final value. But this new feature will show you the value of each expression evaluation.

9. Management Portal Updates

Instead of having to go to aka.ms/logicappsupdate, you’ll be able to get to the list of updates directly from the Management Portal.

10. Export Logic App to Visual Studio

A long-awaited feature, you’ll be able to select a deployed Logic App (from Visual Studio) and import that Logic App into a Logic Apps project in Visual Studio. This will appear in the next version of the Logic Apps Tools for Visual Studio. I’m not sure if you’re exporting from Azure, or importing into Visual Studio… 😊.

11. New Connectors in-progress

New connectors are on their way, including:

• Service Bus Sessions (will enable convoy patterns, yay!!)
• Azure Cognitive Vision (OCR)
• SQL Trigger (trigger when new rows are created)
• Oracle EBS (E-Business Suite)
• ProsperWorks (CRM for Google products)
• Benchmark Email (Email marketing services)
• Inoreader (content and news reader)

Whew! And that’s it.

Don’t forget the Global Integration Bootcamp, coming to a location near you… https://www.globalintegrationbootcamp.com/