In our Integration projects, especially nowadays on Azure Integration Services, we often work with JSON messages. And despite Logic App Designer allowing us to generate a JSON schema based on a JSON payload (or JSON message), but there are many situations we need the opposite! We want to generate a dummy message from the JSON Schema in order to test our Logic Apps solutions or now our Data Mapper maps.
Unfortunately, there aren’t any out-of-the-box capabilities inside Logic Apps or the new Data Mapper to generate a dummy message.
I’m doing a lot of demos and sessions on the new Data Mapper, and you cannot imagine the times I forget the input message for each specific map, and I find myself constantly looking for that information.
In order to accomplish that, we usually use free online tools like this one: https://www.liquid-technologies.com/online-schema-to-json-converter, but what annoys me about these online tools is that all data are stored in their log files, and sometimes for privacy/security concerns that can be a problem. Mainly because of this reason, I decide, along with my team, to start creating these free tools with the source code available on GitHub in order to not raise the same suspicions.
Create a sample JSON document from a JSON Schema Tool
Create a sample JSON document from a JSON Schema Tool is a straightforward Windows tool that allows you to create a JSON Document based on the JSON Schema we provide. No data are stored in log files.
In this new version, we will be able to load a JSON message from our local hard drive.
Download
Hope you find this useful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Credits
Luis Rigueira | Member of my team and one of the persons responsible for developing this tool.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
In our Integration projects, especially nowadays on Azure Integration Services, we often work with JSON messages. And despite Logic App Designer allowing us to generate a JSON schema based on a JSON payload (or JSON message), there are many situations that we find ourselves in the need to create a JSON Schema outside the Logic App. One good example is when we are creating a map inside the new Data Mapper.
Unfortunately, there aren’t any out-of-the-box capabilities inside the new Data Mapper to generate a JSON Schema from a JSON Message, so we only have two options:
Create a dummy Logic App to generate the JSON Schema and then manually copy it and save in the Artifacts folder;
Or we will use free online tools like this one: https://www.liquid-technologies.com/online-json-to-schema-converter, but what annoys me about these online tools is that all data are stored in their log files, and sometimes for privacy/security concerns that can be a problem.
Mainly because of these reasons, I decide, along with my team, to start creating these free tools with the source code available on GitHub in order to not raise the same suspicions.
JSON Document to JSON Schema Converter Tool
JSON Document to JSON Schema Converter Tool is a straightforward Windows that allows you to generate a JSON Schema based on the sample JSON Document that we provide. No data are stored in log files.
Where can I download
Hope you find this useful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Credits
Luis Rigueira | Member of my team and one of the persons responsible for developing this tool.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
In our Integration projects, especially nowadays on Azure Integration Services, we often work with JSON messages. And despite Logic App Designer allowing us to generate a JSON schema based on a JSON payload (or JSON message), but there are many situations we need the opposite! We want to generate a dummy message from the JSON Schema in order to test our Logic Apps solutions or now our Data Mapper maps.
Unfortunately, there aren’t any out-of-the-box capabilities inside Logic Apps or the new Data Mapper to generate a dummy message.
I’m doing a lot of demos and sessions on the new Data Mapper, and you cannot imagine the times I forget what the input message for each specific map is, and I find myself always looking for that information.
In order to accomplish that, we normally use free online tools like this one: https://www.liquid-technologies.com/online-schema-to-json-converter, but what annoys me about these online tools is that all data are stored in their log files, and sometimes for privacy/security concerns that can be a problem. Mainly because of this reason, I decide, along with my team to start creating these free tools with the source code available on GitHub in order to not raise the same suspicions.
Create a sample JSON document from a JSON Schema Tool
Create a sample JSON document from a JSON Schema Tool is a straightforward Windows tool that allows you to create a JSON Document based on the JSON Schema that we provide. No data are stored in log files.
Download
Hope you find this useful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Credits
Luis Rigueira | Member of my team and one of the persons responsible for developing this tool.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
We just released another version of our Azure Function JSON Schema Validation, adding support to another feature. In this case, a very basic one, required fields.
In order to specify the mandatory properties or elements, we need to use the required keyword, where you can specify a list of strings that need to be present as key names in the list of key:value pairs that appear in a JSON document. Each of these strings must be unique.
JSON Schema Validation Function
The JSON Schema Validation is a simple Azure Function that allows you to validate your JSON message against a JSON Schema, enabling you to specify constraints on the structure of instance data to ensure it meets the requirements.
The function receives a JSON payload with two properties:
Or a 400 Bad Request if there are validation errors/issues.
Where can I download it?
You can download the complete Azure Functions source code here:
Hope you find this helpful! So, if you liked the content or found it helpful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Big thanks to my team members Luís Rigueira and Diogo Formosinho for testing and adding this new feature.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
We just released a new version of our Azure Function JSON Schema Validation, adding support for more complex schema validations. In this case, we add support for applying subschemas validation conditionally.
The if, then and else keywords allow the application of a subschema based on the outcome of another schema, much like the if/then/else constructs you’ve probably seen in traditional programming languages.
If if is valid, then must also be valid (and else is ignored.) If if is invalid, else must also be valid (and then is ignored).
If then or else is not defined, if behaves as if they have a value of true.
If then and/or else appear in a schema without if, then and else are ignored.
JSON Schema Validation Function
The JSON Schema Validation is a simple Azure Function that allows you to validate your JSON message against a JSON Schema, enabling you to specify constraints on the structure of instance data to ensure it meets the requirements.
The function receives a JSON payload with two properties:
Hope you find this helpful! So, if you liked the content or found it helpful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Big thanks to my team member Luís Rigueira for adding this new feature.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
JSON Schema is a declarative language that allows you to annotate. It provides a format for what JSON data is required for a given application and how to interact with it and validate JSON documents to ensure it meets the requirements.
Applying JSON Schemas validation in your solutions will let you enforce consistency and data validity across similar JSON data.
If you are not familiar with JSON Schema, you will then notice that the JSON Schema itself is written in JSON-based format. It’s just a declarative format for “describing the structure of other data”. This is both its strength and its weakness (which it shares with other similar schema languages). It is easy to concisely describe the surface structure of data, and automate validating data against it. However, since a JSON Schema can’t contain arbitrary code, certain constraints exist on the relationships between data elements that can’t be expressed. JSON Schema is a proposed IETF standard.
JSON Schema Validation Function
The JSON Schema Validation is a simple Azure Function that allows you to validate your JSON message against a JSON Schema, enabling you to specify constraints on the structure of instance data to ensure it meets the requirements.
The function receives a JSON payload with two properties:
Or a 400 Bad Request if there are validation errors/issues.
Where can I download it?
You can download the complete Azure Functions source code here:
Hope you find this helpful! So, if you liked the content or found it helpful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Big thanks to my team member Diogo Formosinho for testing and helping me develop this function with me!
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
Another common task for us developers inside Azure Integration Services, especially inside Logic Apps, is manually creating a JSON message, either inside a Parse JSON action, Compose action, or directly in the connectors (or any other way), but there is a catch…
If you work in the Azure Portal and create an invalid JSON message inside an action or connector, the editor will not allow you to save the Logic App. Instead, it will give you an error saying that the definition contains invalid parameters.
However, if you are creating the same Logic App Consumption inside Visual Studio:
We can successfully validate this Logic App:
And we can actually successfully deploy this Logic App:
And that will become a problem once we run our Logic App. For this reason, it is always good for you to guarantee that the JSON message is well formatted before you deploy your business processes.
And yes, I know many only tools exist to perform this task, so why a Windows tool? Again, for the same reasons I described in my previous tools: security and privacy.
I’m starting to become a freak in terms of security. Nothing is free, and the problem with these online tools is that we never know behind the scenes what they are doing. Are you sure that they are not keeping logs of the inputs we provide and the result outputs? And don’t say, but Sandro, this is just a simple message. Well, many messages have sensitive (private) information from users or companies that sometimes you are not aware of, so it is better to play safe than sorry. It is wise to be careful now so that problems do not occur later on and protect yourself against risk rather than be careless.
JSON Validator Tool
JSON Validator Tool is a lightweight Windows tool that allows you to validate and reformat a JSON message.
To not raise the same suspicions about this tool, the source code is available on GitHub!
Download
Hope you find this useful! So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego!
Credits
Diogo Formosinho | Member of my team and one of the persons responsible for developing this tool.
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
Today while my team and I were trying to do a small Logic App proof-of-concept using a Request – When a HTTP request is received trigger – something quite simple and basic – with a simple JSON payload like:
We were always getting the following error when we were trying to trigger the Logic App using Postman:
InvalidTemplate. Unable to process template language expressions for action ‘For_each’ at line ‘0’ and column ‘0’: ‘The template language expression ‘triggerBody()?[‘Data’]’ cannot be evaluated because property ‘Data’ cannot be selected. Property selection is not supported on values of type ‘String’. Please see https://aka.ms/logicexpressions for usage details.’.
To be honest, I was getting annoyed because this a simple stuff that I have done thousands of times!
Cause
Unfortunately, I cannot use the common excuse: “sorry, it is Friday!” because today is Tuesday :). But I can always say that most of the time the error resides between the chair and the keyboard! ?
You may pay more attention to the Logic App designer when you define a JSON Schema in your When a HTTP request is received trigger. It will warn you not to forget to include a Content-Type header set to application/json in your request!
If you do not provide the Content-Type header, it will assume that is plain text, and it will not parse the JSON and render all the properties, so it will not going to be tokenized, and there you will get this or similar errors.
Solution
The solution is quite simple, on the Postman request add a Content-Type header and set it to application/json.
If you try again… problem is solved!
Hope you find this useful!
My youngest son (almost 5 years old) is a big Star Wars fan, and when I’m trying to write something, and he arrives home and want to play with me, I always say to him: Let the father finish work to earn some money so we can buy more toys… now I promise him that all contributions from my blog are going to be used for buying Star Wars Legos!
So, if you liked the content or found it useful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego! ?
Author: Sandro Pereira
Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.
He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira
BizTalk Server 2020 – 20 days, 20 posts – day 17. I have to confess I didn’t try yet the default JSON Encoder, but I trust this component is still useful in this new release, so today we are migrating the custom JSON Encoder component available in BizTalk Pipeline Components Extensions UtilityPack project: JSON Encoder Pipeline Component.
Carry SOAPHeader To WCF-BasicHttp Pipeline Component
This custom JSON Encoder Pipeline Component is a pipeline component for BizTalk Server which can be used in a Send Pipeline (Encode stage) to encode any XML message into a JSON equivalent in a simple and effective way. It is also 100% compatible with the default JSON Encoder component provided by Microsoft.
This component will allow you to choose between using the default pipeline component provide by Microsoft, at least the internal behavior because I build this pipeline component as an extension of the default JSON Encoder pipeline component, or use the custom behavior that I created. So, from the BizTalk Administration console you will be able to decide if:
You want to use
the behavior of the default JSON Encoder pipeline component provide by
Microsoft by setting the UseCustomEncoder property to False;
or use the custom
behavior to generate the JSON message by setting the UseCustomEncoder property
to True;
To use this pipeline component in your projects you just copy the CustomJSONEncoder.dll file into the Pipeline Components folder that exists in the BizTalk Server installation directory:
You do not need to add this custom pipeline component to be used by the BizTalk Runtime to the Global Assembly Cache (GAC).
What is BizTalk Pipeline Components Extensions Utility Pack?
BizTalk Pipeline Components Extensions Utility Pack is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will provide an extension of BizTalk out-of-the-box pipeline capabilities.
At the beginning of this week I ask on twitter what should be the topic of my next blog posts: BizTalk Server and JSON support or Azure Logic Apps. And the majority of the replies was: BizTalk Server and better JSON support.
For that reason, I will respect my reader’s decision and I will be releasing today a new addition to my BizTalk Pipeline Components Extensions Utility Pack community project for BizTalk Server 2016: Custom JSON Encoder Pipeline Component.
You may be asking yourself: this component already exists by default in BizTalk Server 2016, so why we need another?
Yes, Microsoft provides out-of-the-box a JSON Encoder to transform the XML messages into a JSON message and sends it out. But, unfortunately, in some cases, it doesn’t work as you should expect to work and it will fail, in my case with this error:
Reason: Value cannot be null. Parameter name: key
And in reality what I just want is a simple way, that, I hope, it would work for all scenarios (aka XML messages) in a simple and effective way.
JSON Encoder Pipeline Component
The Custom JSON Encoder is a pipeline component for BizTalk Server which can be used in a Send Pipeline (Encode stage) to encode any XML message into a JSON equivalent in a simple and effective way.
Because I would like to have a choice between using the default pipeline component provide by Microsoft, at least the internal behavior, this pipeline component is an extension of the default JSON Encoder pipeline component and fully compatible with it and you will be able from the BizTalk Administration console you will be able to decide if:
You want to use
the behavior of the default JSON Encoder pipeline component provide by
Microsoft by setting the UseCustomEncoder property to False;
or use the custom
behavior to generate the JSON message by setting the UseCustomEncoder property
to True;
What is BizTalk Pipeline Components
Extensions Utility Pack?
BizTalk Pipeline Components Extensions Utility Pack is a set of custom pipeline
components (libraries) with several custom pipeline components that can be used
in received and sent pipelines, which will provide an extension of BizTalk
out-of-the-box pipeline capabilities.