In a general and abstract way, we can say that Data Mapper maps are graphical representations of XSLT 3.0 (Extensible Stylesheet Language Transformation) documents that allow us to perform simple and visual transformations between XML and JSON messages (more formats will be added in the future). But in reality, Data Mapper maps are composed of two files: a Map Definition file (.yml) that basically is an abstraction of the underlying implementation. And one XSLT file (.xslt) contains all the transformation rules – This is the file that in runtime will be executed by the Logic Apps (standard) engine.
What types of transformation can we apply in Logic Apps?
Similar to BizTalk Server, we can define two types of transformations:
Syntax Transformations: In BizTalk Server, this type of transformation occurs in receive or send pipelines and aims to transform a document into another representation, e.g., CSV to XML. In Logic Apps, that will occur inside the Logic Apps by using an Integration Account or certain actions and expressions depending on which syntax transformation we need to perform. Here the document maintains the same data (semantics) but changes the syntax that is represented. I.e., we translate the document, but typically we don’t modify the structure. Normally, this type of transformation is bidirectional, since we still have the same semantic content, we can apply the same transformation logic and obtain the document in its original format.
Semantic Transformations: In BizTalk Server, this type of transformation usually occurs only in BizTalk maps. Inside Logic Apps (Standard), they will take place in the Data Mapper. Here the document can maintain the same syntax that is represented (XML or JSON) – not mandatory since Data Mapper supports at the moment JSON Schemas and XML Schemas – but changes its semantics (data content). These types of transformations are typically one-way. Since we add and aggregate small parts of the information that compose the document into another document, we may miss important details for its reconstruction.
Introduction to the Data Mapper
The Data Mapper enables us to perform transformations of complex messages visually and effortlessly, graphically represented by links that define the relationships between the various elements of messages.
These relationships between elements are internally implemented as XSL Transformations (XSLT – Extensible Stylesheet Language Transformation), the standard Worldwide Web Consortium (W3C) recommended to perform message transformations.
Data Mapper is a Visual Code extension that allows you to perform mapping transformations. Currently, it has its own extension, but that will be merged with the main extension in the future.
Essentially the editor consists of three main modules:
Source Schema view: this is the data structure of the source message and is on the left side of the main window;
Destination Schema view: this is the data structure of the target message and is on the right side of the main window; the links that define the mapping lead into the destination schema tree view from the grid view, and ultimately from the source schema tree view.
Mapping areaview: is in the middle of the main window, between the two data structures (source and target); This area plays a critical role in the definition of maps, containing the links and Functions that control how data in a source instance message is transformed into an instance message that conforms to the destination schema. The mapping area can have multiple layers, each of them associated with a specific record on the destination schema, allowing us this way to organize complex maps in a better way.
Apart from these three modules, there are other important windows for the developer:
Function panel: At the left side of the source schema, providing access to all Functions we can use in the Data Mapper.
Function properties panel: in this panel, we can see and modify the properties of a selected function inside the Data Mapper.
Task List and Output windows: much of the time hidden, we can and must use these windows to examine the results of saving and testing we do in our maps. These windows normally appear underneath the Mapping area or the Function properties panel.
Code view Panel: On this panel, you can view Map definitions rules (abstraction of the underlying implementation). It is a read-only panel.
Basic maps functionalities (Document mapping)
There are several mapping functionalities that you can perform inside maps, like Data normalization, Transform Injection (XLST injection), calculating values covering math and scientific functoids, and so on. Still, most of the time, transformations are quite simple, and when we perform a transformation in the message 5 basic functionalities typically arise:
Simple mapping of a given value (direct copy)
Concatenation of values
Conditional selection
Custom scripts
Add new values (data)
Here we will take a simple transformation problem that will address each one of these functionalities or operations, providing a simple example of how to accomplish this inside the Data Mapper. Of course, there are plenty of other operations or transformation rules that we could do that will not be addressed here.
Hope you find this helpful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
Function Chain? What is a Function Chain? And how that works inside the Data Mapper?
When we use more than one cascading function to perform a transformation rule, we call it a Function Chain, or a chain of functions that are going to be executed in order. This way, we can apply more complex mapping rules inside the Logic Apps (Standard) Data Mapper. By the way, we had the same concept inside BizTalk Server Maps and Logic App Consumption Map (that is, the BizTalk Server Maps extracted and isolated)
The picture below shows us a Function Chain inside the Data Mapper. In this case, we are calculating national and international calls based on the phone number:
In this video, we will learn t what a Function Chain is inside the Logic Apps (Standard) Data Mapper and how it works.
Hope you find this helpful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
Unfortunately, no Logic App connector can make the bridge to RabbitMQ, which makes this integration challenge a little bit more complicated. However, we have the ability to create an Azure Function by using the RabbitMQ trigger for Azure Functions to overpass this limitation.
And we saw and explained in our last blog post that Azure Functions integrates with RabbitMQ via triggers and bindings. The Azure Functions RabbitMQ extension allows you to send and receive messages using the RabbitMQ API with Functions.
The purpose of this video is to explain how you create a POC that allows you to receive a message in a RabbitMQ queue, and that event triggers the Azure Function that then will route the message to a Logic App.
This was a real problem presented by a client during one of our Logic Apps training courses, where they have RabbitMQ on-premises, and they did want to pull messages from a queue into a Logic App Consumption to integrate them with other systems.
Hope you find this helpful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Big thanks to my team member Luís Rigueira for creating this video.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
In my last blog post, I explain in detail the String functions available in the new Data Mapper, and I endup documenting each of them.
String functions are used to manipulate strings in standard ways, such as conversions to all uppercase or all lowercase, string concatenation, determination of string length, white space trimming, etc. If you come from the BizTalk Server background or are migrating BizTalk Server projects, they are the equivalent of String Functoids inside BizTalk Mapper Editor.
The String functoids are:
Codepoints to string: Converts the specified codepoints value to a string and returns the result.
Concat: Combines two or more strings and returns the combined string.
Contains: Returns true or false based on whether the string input contains the specified substring.
Ends with: Returns true or false based on whether the string input ends with the specified substring.
Length: Returns the number of items in the specified string or array.
Lowercase: Returns a string in lowercase format.
Name: Returns the local name of the selector node, which is useful when you want to retrieve the name of the incoming message component, not the value.
Regular expression matches: Returns true or false based on whether the string input matches the specified regular expression.
Regular expression replace: Returns a string created from the string input by using a given regular expression to find and replace matching substrings with the specified string.
Replace: Replaces a substring with the specified string and return the new complete string.
Starts with: Returns true if the given string starts with the specified substring.
String to codepoints: Converts the specified string to codepoints.
Substring: Returns characters from the specified string, starting from the specified position.
Substring after: Returns the characters that follow the specified substring in the source string.
Substring before: Returns the characters that precede the specified substring in the source string.
Trim: Returns the specified string with all the leading and trailing white space characters removed.
Trim left: Returns the specified string with all the leading white space characters removed.
Trim right: Returns the specified string with all the trailing white space characters removed.
Uppercase: Returns a string in uppercase format.
In this video, we will see each of these String Functions in action. For each one, we will provide simple input data, and we will see what the expected output is.
Hope you find this helpful! So, if you liked the content or found it useful and want to help me write more, you can buy (or help buy) my son a Star Wars Lego!
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
Today is a great day to be a BizTalk Developer! Why? Microsoft has introduced .Net 4.0, Windows Application Server (Dublin), and Microsoft’s Modeling Platform (Oslo). These new technologies compliment the existing Connection System technologies.
As you look though the various technologies one thing is consistent: Take the best things of BizTalk and allow others outside of BizTalk to leverage it!
I’ve put together over an hour of first look videos to help show the new features of Microsoft’s latest offerings.
The following videos are now available on BizTalkGurus.com:
First Look at Windows Application Server (Dublin) – If you do not watch anything else, watch this video! This video takes a look at the new Windows Application Server features. It shows how to export existing applications and import them into another host on another server, how to set variables inside Workflow 4.0 to be tracked, how to create a custom tracking profile, and how to configure your service to use this custom profile. (Download WMV)
First Look at Quadrant – Oslo’s Modeling Tool -This video is a quick overview of the new modeling tool Quadrant. It walks though the basic user experience. It shows how to work with and edit exiting models, drop workpads onto the workflow surface, add and edit data, and edit existing models. (Download WMV)
First Look at M – Oslo’s Modeling Language – This video walks though creating a simple model using the new M Modeling Language. A simple Domain-Specific Language (DSL) model is also looked at. How often do you get to see a new language? (Download WMV)
Consuming WCF Services in Workflow 4.0 – This video takes a quick look at the new designer experience for Workflow 4.0 inside Visual Studios 10. A simple application is created to communicate with an existing WCF Service. This demonstrates the new interface for Workflow and WCF interactions. (Download WMV)
Flowcharts and Rules in Workflow 4.0 – This video walks though creating an application using the new Flowchart style of workflow. Inside the flowchart workflow, the new Workflow 4.0 Rules are used to evaluate input data. (Download WMV)
It is important to point out that these sets of technologies are not designed to replace BizTalk but rather enhance the rest of the framework. BizTalk still serves a mission critical need as the Microsoft Integration Server and Adapter provider.
Note: This video is based on BizTalk Server 2006 R2 BETA
This nearly 13 minute video gives a quick, high level First Look at BizTalk Server 2006 R2 WCF Adapter using the Basic HTTP Binding. This is intended to show how easy it is to use BizTalk 2006 R2 and WCF to build applications.
Note that this is based on Beta code and is subject to change in future releases.
This content should be current for the RTM release of BizTalk 2006 R2 and beyond.
This 9:07 minute video covers one of the new features of Biztalk 2006, Recoverable Interchange Processing. This is the ability to fail a single message inside an interchange without failing the whole interchange. This works with both the Xml Disassembler and the Flat File Disassembler.
This video was made for BizTalk 2006 but should still be relevant for other versions of BizTalk.
This 9:49 minute video covers the three different output options from the biztalk flat file Disassembler provided in Biztalk. The three options are single detail records without headers or trailers, all detail records together without header and trailers, or all details together with the header and trailer.
This video was made for BizTalk 2004 but should still be relevant for other versions of BizTalk.
I have put together a short 10 minute biztalk training video covering some of the different output options available using the Flat File Disassembler in Biztalk Server 2004.
This video covers three different output options from the flat file disassembler. The three options covered are single detail records without a header or trailer, all detail records together without a header or trailer, or all details together with the header and trailer.
This video walks through the Flat File Disassembler Sample from a past post. It might be helpful to have the sample downloaded before watching the video.
I have the video available for live play via shock wave or downloadable via Windows Media Player.
Watch the video: Flat File Disassembler Output Options Video
