Your Guide to BizTalk Message Context Properties whitepaper

Your Guide to BizTalk Message Context Properties whitepaper

As you most likely are aware, when a document is received by a BizTalk Server adapter, the adapter creates a BizTalk message for the document. The BizTalk message contains the document that was received as well as a message context. The message context is a container for various properties that are used by BizTalk Server when processing the document. Each property in the Message Context is composed of three things, a name, a namespace, and a value.

Message context properties are added to the message context throughout the lifetime of the message as it passes through the BizTalk Server. These properties:

  • Are either extracted from the message itself, for example, order id or shipment number
  • Or added by pipelines and adapters at the receive location, for example, transport name or receive port name

There are mainly two benefits of this context:

  • The first is to provide the various components of BizTalk, an easy access to these properties, without having to parse the message
  • The second is to support content-based routing.

There are two different types of message context properties used by BizTalk as described below:

  • Distinguished Fields
  • Property Fields.

In this whitepaper, you will learn the key differences between Distinguished and Property Fields but most importantly you will have access to a complete list of known property schema and properties used internally by the BizTalk Server out-of-the-box components.

What’s in store for you?

This whitepaper will give you a detailed understanding of the following:

  • BizTalk Message Context Properties
    • What are Property fields and how to promote properties?
    • What are distinguished fields and how to create a distinguished field?
    • Summary of differences between Property Fields and Distinguished Fields
  • System property schema and properties
  • Error Report property schema and properties
  • Legacy schema and properties
  • Microsoft BizTalk XLANGs BTXEngine schema and properties
  • Message Tracking schema and properties
  • BizTalk Framework Schema and Properties
  • MIME-SMIME Property Schema and Properties
  • XML and Flat File Property Schema and Properties
  • File adapter property schema and properties
  • FTP Adapter Property Schema and Properties
  • HTTP Adapter Property Schema and Properties
  • MQSeries Adapter Property Schema and Properties
  • MSMQ Adapter Property Schema and Properties
  • POP3 Adapter Property Schema and Properties
  • SMTP Adapter Property Schema and Properties
  • SFTP Adapter Property Schema and Properties
  • SOAP Adapter Property Schema and Properties
  • SQL Adapter Property Schema and Properties
  • WCF Adapters Property Schema and Properties
  • Windows SharePoint Services Adapter Property Schema and Properties
  • Azure Service Bus Adapter Property Schema and Properties
  • Azure Blob storage Adapter Property Schema and Properties
  • Azure Event Hubs Adapter Property Schema and Properties
  • Office 365 Outlook Calendar Adapter Property Schema and Properties
  • Office 365 Outlook Contact Adapter Property Schema and Properties
  • Office 365 Outlook Email Adapter Property Schema and Properties
  • SharePoint Online Adapter Property Schema and Properties
  • Microsoft BizTalk Accelerator for HL7 (BTAHL7) Property Schema and Properties
  • How to write or promote properties in the context of a messages through the BizTalk API
    • Promote method
    • Write method
    • Property Schema Base

Where I can download it

You can download the whitepaper here:

I hope you enjoy reading this paper and any comments or suggestions are welcome.

Azure Logic Apps team is interested in your feedback – BizTalk Customer Survey

Azure Logic Apps team is interested in your feedback – BizTalk Customer Survey

Azure Logic Apps team is looking to learn about your BizTalk architectures and scenarios to better understand how Microsoft can best address future integration needs within the Azure Integration Services platform.

No matter if you are considering migrating in the future to Azure Integration Services or staying on-premises for a couple of more years (or forever) this is a great opportunity to provide feedback to the Microsoft Integration team and positively influence the outcome of new features.

So, for customers that are using BizTalk Server or were using BizTalk Server in the past, this survey is for you! If you are a consulting company providing services to clients using BizTalk Server, this survey is also for you! The Azure Logic Apps team would love to hear from you!!! They want to enter and poke around your brain, your world and gather all the feedback regarding BizTalk Server:

  • What version you are using or were using?
  • How many applications do you have?
  • How many orchestrations, receive locations, and send ports do you have?
  • What features you are using?
    • If you are like me, all! ? depending on the client ? I even select others ?
  • Which connectors are being used in your BizTalk Server implementation?
  • Do you have any custom adapters?
  • What Integration Patterns are you currently using?
  • and many other simple questions

Don’t complain in the future about the lack of features that will suit you better. The team is interested in how they can best support customers transitioning integration workloads to Azure Integration Services (AIS)… you are not going to move to Azure? The survey does not take that long to respond to, so if you are using BizTalk Server try to respond nevertheless, by doing that you can help other customers on their journey.

For the bad mouths of this universe or for those who like to create rumors because they have nothing to say… this survey does not mean that BizTalk Server is dead! The goal is to help Microsoft prioritize upcoming investments in Azure Integration Services (AIS).

I did my part!

Please fill out the following survey to help Azure Logic Apps: https://lnkd.in/gVdSsRPt#azure#iPaaS#middleware

BizTalk Server CI/CD from zero to hero whitepaper

BizTalk Server CI/CD from zero to hero whitepaper

Historically, deploying BizTalk Server solutions across environments is or can be a complicated process depending on how complex is your solution. There are many ways to deploy BizTalk artifacts for example:

  • Importing them as part of an application by using the Deployment Wizard (from a .msi file), importing them using BTSTask.exe – this is the default way to deploy across environments.
    • You can replace and use allow BTSTask, and PowerShell scripts.
  • Or deploy them from Visual Studio – this is the default way to deploy to your development environment.

Throughout the years, the BizTalk Server Community created an open-source deployment framework called Deployment Framework for BizTalk (BTDF) – https://github.com/BTDF/DeploymentFramework. The Deployment Framework for BizTalk is an easy-to-use toolkit for deploying and configuring your BizTalk solutions. In reality, BTDF is an MSBuild project with custom MSBuild tasks and it can be customizable according to customer BizTalk project needs, it is also extensible. This framework brings new capabilities and advantages to deploying BizTalk Server solutions, but it also has limitations or disadvantages.

Microsoft has introduced automated deployment of BizTalk Applications in BizTalk Server 2016 Feature Packs using Azure DevOps (previously called Visual Studio Team Services – VSTS). In BizTalk Server 2016 Feature Pack 1, automatic deployment and application lifecycle management (ALM) experience was introduced. The automatic deployment process has been improved with the release of BizTalk Server 2016 Feature Pack 2. These features were only available on the Enterprise edition of BizTalk Server 2016.

BizTalk Server 2020 brings all these functionalities out-of-the-box across all editions: Enterprise, Standard, Development, or Branch.

To accomplish this, we need basically 3 main steps:

  • BizTalk Server: Add a BizTalk Server Application project to your Visual Studio solution.
  • DevOps: Create a build agent.
  • DevOps: Create a Build and release Azure Pipeline.

This whitepaper will address and explain how you can implement CI/CD oriented to BizTalk Server using Azure DevOps Pipelines.

In this whitepaper, Pedro Almeida and I will provide a detailed introduction to CI/CD. It teaches how to Create a project collection. Learn how to prepare the visual studio for projects end to end. A well-defined pipeline. Helps you understand how to save development time by thinking long-term since it is a low-cost, high-return scenario.

What’s in store for you?

This whitepaper will give you a detailed understanding of the following:

  • An introduction to:
    • What is a CI/CD Pipeline?
    • What are CI/CD Pipelines?
    • What is Azure DevOps?
  • Create an organization or project collection in Azure DevOps
  • Create a project in Azure DevOps
  • Preparing your Visual Studio BizTalk Server project for CI/CD
    • Creating a BizTalk Server Deployment Project
      • Add the application project
      • Making your Bindings dynamic for deployment
      • Configure the BizTalkServerInventory JSON template
    • Publish your code
  • Create a Personal Access Token
    • Install the Build Agent
  • Building your Azure Pipeline
    • Building the Pipeline
    • Building the Release Pipeline
    • Defining the Variables
  • Using SSO Application Configuration with CI/CD

Where I can download it

You can download the whitepaper here:

I hope you enjoy reading this paper and any comments or suggestions are welcome.

Logic App CI/CD from zero to hero whitepaper

Logic App CI/CD from zero to hero whitepaper

Continuous Integration and Continuous Deployment (CI/CD) is a practice that has become an essential aspect of Azure development. Although it is possible to execute each of the CI/CD pipeline steps manually, the actual value can be achieved only through automation.

And to improve software delivery using CI/CD pipelines, either a DevOps or a Site Reliability Engineering (SRE) approach is highly recommended.

In this whitepaper, Pedro Almeida and I will demonstrate how you can use Azure DevOps Pipelines to implement CI/CD based on Logic Apps (consumption).

We will explain it all in detail, from creating a project in Azure DevOps, and provisioning a Logic App Consumption to configuring the built Logic App for CI/CD.

What’s in store for you?

This whitepaper will give you a detailed understanding of the following:

  • An introduction to:
    • What is a CI/CD Pipeline?
    • What are CI/CD Pipelines?
    • What is Azure DevOps?
  • Create an organization or project collection in Azure DevOps
  • Create a project in Azure DevOps
  • Building a Logic App (Consumption) from scratch
  • Setting up the Visual Studio Logic App (Consumption) project for CI/CD
  • A step-by-step approach to building Azure Pipelines

Where I can download it

You can download the whitepaper here:

I hope you enjoy reading this paper and any comments or suggestions are welcome.

AZUGPT | April 26, 2022 | Event-driven apps in Azure / Making sense of unstructured data with AI

AZUGPT | April 26, 2022 | Event-driven apps in Azure / Making sense of unstructured data with AI

We are thrilled to be back doing a presential event on our Azure User Group Portugal! It has been so long! And even happier to be in Porto City.

Azure User Group Portugal is a user group for anyone interested in Cloud Computing with a great focus on Microsoft Azure. If you work with or have an interest in the Microsoft Cloud this is the user group to attend and follow our events. Join our group to keep posted about all the new meetings. Looking forward to having you as a member.

In this April 2022 edition, we will be having two distinguished speakers: Laurent Bugnion (Principal Cloud Developer Advocate at Microsoft) and Henk Boelman (Cloud Advocate at Microsoft) in two sessions about:

  • Event-driven apps in Azure
    Using Azure, it is easier than ever to build event-driven web applications, for example using Azure Functions and the Azure SignalR service. Laurent Bugnion, a Cloud Advocate for Microsoft, will show you how he implemented such a solution to solve a real-world problem. This presentation will dive into a production application called Timekeeper, that Microsoft uses to run some of its live TV shows such as the Hello World daily show. More information about Timekeeper is at http://timekeeper.cloud
  • Making sense of unstructured data with AI
    Do you have a lot of data in unreadable formats such as PDFs, images, and audio files and want the ability to extract this rich information, analyze it, and act on it? In this session, you’ll learn how to combine a set of Cognitive Services like Azure Cognitive Search to make sense of this data in a short amount of time. We’ll discuss AI concepts, like the ingest-enrich-explore pattern, skillsets, leveraging cognitive services, knowledge bases, and connecting all these elements together to build an intelligent search experience into an application.

Event Details and Agenda

The event will start at 6:30 PM (GMT) at the headquarters of DevScope at Rua de Passos Manuel, 223 – 4º Floor – 4000-385 Porto, Portugal.

Agenda:

  • Session 1: Event-driven apps in Azure
  • Session 2: Making sense of unstructured data with AI

Speakers: Laurent Bugnion and Henk Boelman
Event Language: English

Register

Join us and reserve your presence at the AZUGPT April 26, 2022 edition on Tuesday, April 26, 2022, it is free!

Message Archive Pipeline Component

Message Archive Pipeline Component

Indeed, another message archive pipeline component on my BizTalk Pipeline Components Extensions Utility Pack project is available on GitHub!

This time I decided to create a brand new component called the Message Archive Pipeline Component.

For those who aren’t familiar with it, the BizTalk Pipeline Components Extensions Utility Pack project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in receive and sent pipelines. Those pipeline components provide extensions of BizTalk’s out-of-the-box pipeline capabilities.

Message Archive Pipeline Component

The Message Archive Pipeline Component is a pipeline component that can be used to arch incoming/outgoing messages from any adapters into a local or shared folder. It is very identical and provides the same capabilities as the already existing BizTalk Server Local Archive pipeline component:

  • It can be used in any stage of a receive pipeline or send pipeline;
  • It can be used in multiple stages of a receive pipeline or send pipeline;
  • It provides an option for you to specify the location path for where you want to save the message: local folder, shared folder, network folder.
  • It can be used from any adapter:
    • If the adapter provides the ReceivedFileName property promoted like the File adapter or FTP adapter the component will take this value into consideration and save the message with the same name;
    • Otherwise, it will use the MessageID, saving the file with the MessageID has its name without extension.

So what are the differences between them?

The significant differences between these two components are that the Message Archive Pipeline Component allows you to:

  • Set the filename using macros like %datetime%, %ReceivePort%, %Day%, etc.
    • For example, %ReceivePort%_%MessageID%.xml
  • Set the archive file path once again using macros:
    • for example C:BizTalkPortsArchiveARCHIVE%Year%%Month%%Day%
  • If you don’t want to overwrite existing files, you can specify an additional Macro to distinguish them.
    • For example _%time%
  • You can set up this component for high performance using forward-only streaming best practices.
    • In short, this means developing your pipeline components so that they do their logic either as a custom stream implementation or by reacting to the events available to you through the Microsoft.BizTalk.Streaming.dll stream classes. Without ever keeping anything except the small stream buffer in Memory and without ever seeking the original stream. This is best practice from the perspective of resource utilization, both memory and processor cycles.

This is the list of properties that you can set up on the archive pipeline component:

Property Name Description Sample Values
OverwriteExistingFile Define if the archive file is to be overwritten if already exists true/false
ArchivingEnabled Define if the archive capabilities are enabled or disabled true/false
ArchiveFilePath Archive folder path. You can use macros to dynamically define the path. C:Archive%Year%%Month%%Day%
ArchiveFilenameMacro File name template. If empty the source file name or MessageId will be used. You can use macros to dynamically define the filename. %ReceivePort%_%MessageID%.xml
AdditionalMacroIfExists If a file already exists and OverwriteExistingFile is set to false, a suffix can be added. If empty the MessageId will be used. You can use macros to dynamically define this suffix. _%time%
OptimizeForPerformance Setting to apply high performance on the archive true/false

Available macros

This is the list of macros that you use on the archive pipeline component:

Property Name Description
%datetime% Coordinated Universal Time (UTC) date time in the format YYYY-MM-DDThhmmss (for example, 1997-07-12T103508).
%MessageID% Globally unique identifier (GUID) of the message in BizTalk Server. The value comes directly from the message context property BTS.MessageID.
%FileName% Name of the file from which the File adapter read the message. The file name includes the extension and excludes the file path, for example, Sample.xml. When substituting this property, the File adapter extracts the file name from the absolute file path stored in the FILE.ReceivedFileName context property. If the context property does not have a value the MessageId will be used.
%FileNameWithoutExtension% Same of the %FileName% but without extension.
%FileNameExtension% Same of the %FileName% but in this case only the extension with a dot: .xml
%Day% UTC Current day.
%Month% UTC Current month.
%Year% UTC Current year.
%time% UTC time in the format hhmmss.
%ReceivePort% Receive port name.
%ReceiveLocation% Receive location name.
%SendPort% Send port name.
%InboundTransportType% Inbound Transport Type.
%InterchangeID% InterchangeID.

How to install it

As always, you just need to add these DLLs on the Pipeline Components folder that in BizTalk Server 2020 is by default:

  • C:Program Files (x86)Microsoft BizTalk ServerPipeline Components

In this particular component, we need to have this  DLL:

  • BizTalk.PipelineComponents.MessageArchive.dll

How to use it

Like all previous, to use the pipeline component, I recommend you to create a generic or several generic pipelines that can be reused by all your applications and add the Message Archive Pipeline Component in the stage you desire. The component can be used in a stage of the receive and send pipelines.

Download

THIS COMPONENT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download Message Archive Pipeline Component from GitHub here:

Receive Location Name Property Promotion Pipeline Component

Receive Location Name Property Promotion Pipeline Component

I just updated my BizTalk Pipeline Components Extensions Utility Pack project available on GitHub with a new component: Receive Location Name Property Promotion Pipeline Component.

For those who are not familiar, this project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will extend BizTalk’s out-of-the-box pipeline capabilities.

Receive Location Name Property Promotion Pipeline Component

Receive Location Name Property Promotion Pipeline Component is a simple pipeline component to promote the Receive Location Name (ReceiveLocationName) property to the context of the message. Several BizTalk Server context properties are not promoted by default with BizTalk Server, which means that they are not available for routing. 

One such property is the ReceiveLocationName property. While the ReceivePortName property is available in the BTS namespace, the ReceiveLocationName property is not promoted. It cannot be used for routing nor access it from inside an orchestration.

My team and I kept that behavior creates this project as a proof-of-concept to explain how you can promote properties to the context of the message.

Create a Property schema

To promote properties to the context of the message, we will need a Property Schema with these properties. In our case, we will add only one property called: ReceiveLocationName.

Property Name Date Type Property Schema Base
ReceiveLocationName xs:string MessageContextPropertyBase

Note: A MessageContextPropertyBase property means that the XPath may or may not exist. 

Create a Pipeline Component

To actually promote the properties to the context of the message, we need to create a pipeline component to do the trick.

string rlocationname = (string)pInMsg.Context.Read("ReceiveLocationName", "http://schemas.microsoft.com/BizTalk/2003/system-properties");
pInMsg.Context.Promote("ReceiveLocationName", "https://BizTalk.Pipeline.Components.RcvLocationPromotion.PropertySchema", rlocationname);

How to use it

Once you deploy the property schema and a receive pipeline component containing the Receive Location Name Property Promotion Pipeline Component, you can start to apply Content-based routing using the Receive Location Name.

Download

THIS COMPONENT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download Receive Location Name Property Promotion Pipeline Component from GitHub here:

BizTalk PDF2Xml Pipeline Component

BizTalk PDF2Xml Pipeline Component

I just updated my BizTalk Pipeline Components Extensions Utility Pack project available on GitHub with two new components. The first one was the Archive Pipeline Component for BizTalk Server that I blogged on the BizTalk360 blog, and this new one that I will address here is the BizTalk PDF2Xml Pipeline Component.

For those who are pt familiar, this project is a set of custom pipeline components (libraries) with several custom pipeline components that can be used in received and sent pipelines, which will extend BizTalk’s out-of-the-box pipeline capabilities.

BizTalk PDF2Xml Pipeline Component

BizTalk PDF2Xml Pipeline Component is, as the name mentioned, a decode component that transforms the content of a PDF document to an XML message that BizTalk can understand and process. The component uses the itextsharp library to extract the PDF content. The original source code was available on the CodePlex (pdf2xmlbiztalk.codeplex.com). Still, I couldn’t validate who was the original creator. So, the component first transforms the PDF content to HTML, and then using an external XSLT, will apply a transformation to convert the HTML into a know XML document that BizTalk Server can process. 

My team and I kept that behavior, but we extended this component and added the capability also to, by default, convert it to a well know XML without the need for you to use an XSTL transformation directly on the pipeline.

How does this component work?

This is the list of properties that you can set up on the PDF2XML pipeline component:

Property Name Description Sample Values
InternalProcessToHTML Value to decide if you want the component to transform the PDF content to HTML or XML True/False
IsToApplyTrasnformation Value to decide if you want to apply a transformation on the pipeline component or not True/False
XsltFilePath Path to an XSLT transformation file C:transfmymap.xslt

Once you pass the PDF by this component and depending on how you configure it, the outcome can be:

  • All PDF content in an HTML format;
  • All PDF content in an XML format;
  • Part of the PDF content on an XML format (if you apply a transformation)

Unfortunately, on my initial tests, this component works well with some PDF files, but others simply ignore its content. Nevertheless, I make it available as a prof-of-concept.

Download

THIS COMPONENT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download BizTalk PDF2Xml Pipeline Component from GitHub here:

Best Practices while working with Schemas Pattern restrictions

Best Practices while working with Schemas Pattern restrictions

Yesterday I spoke about how you can apply custom pattern restrictions to properly validate DateTime, Date, and Time formats inside elements or attributes. You can see more about it here:

And I mention that these Regular Expressions can be simple and relatively easier to read if you have some basic knowledge like this one below:

  • d{4}d{2}d{2} – that is a simple format that expects 4 digits followed by 2 additional digits and another 2 digits that is the date in the YYYYMMDD format without validating the accuracy of months or days and where:
    • YYYY is the for digits year, like 2022.
    • MM is the 2 digits month, like 01.
    • DD is the 2 digits day, like 21.

But it can get highly complex that even people with good knowledge have difficulty translating the expression to the expected pattern, like the one below:

  • ^(?:(?:31(/|-|.)(?:0?[13578]|1[02]))1|(?:(?:29|30)(/|-|.)(?:0?[1,3-9]|1[0-2])2))(?:(?:1[6-9]|[2-9]d)?d{2})$|^(?:29(/|-|.)0?23(?:(?:(?:1[6-9]|[2-9]d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1d|2[0-8])(/|-|.)(?:(?:0?[1-9])|(?:1[0-2]))4(?:(?:1[6-9]|[2-9]d)?d{2})$

It is not simple to look to this RegEx and say: “yep, we are expecting this Date format: MM/DD/YYYY like 12/11/1999, and by the way, it validates the Leap Year!”

Imagine users that don’t have strong know-how about RegEx!

So, what can we do to improve this experience? What are the best practices in these cases?

Best practices

A good best practice to improve readability while documenting your schemas is to add notes to these elements or attributes.

We can and should use the Notes property to enter notes, such as comments related to the business process, that you would like to make about the selected RecordField Element, or Field Attribute node. In these DateTime, Date, and Time cases, we can simply add the expected format in the Notes property, like:

  • Format: YYYY-MM-DD
  • Format: HH:mm:ss
  • Format: YYYYMMDD
  • Format: HHmm
  • Format: YYYY-MM-DD – It validates Leap Year

To accomplish this, we need to

  • Right-click on the Element or Attribute fields in the schema tree view and select the Properties option.
  • On the Properties window, click on the … (three dots) on the Notes property. This action will open a Notes window where you can add all your relevant notes.

Do that to all your elements and fields that are using pattern restrictions.

Even non-technical guys can understand the Schema specification you provide or are consuming. This best practice implementation will also help you improve productivity since you will not spend too much time decompiling Regular Expressions.

Download

THIS SAMPLE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the POC: BizTalk Schemas Handle Restrictions on Date from GitHub here:

BizTalk Schema Validation: DateTime Restrictions

BizTalk Schema Validation: DateTime Restrictions

Today I was involved in a BizTalk Schema importation that includes not-so-used restrictions on Date and Time elements formats. And that gave me the idea and inspiration to create this blog post.

When we work with DateTime on Schemas by default we can choose from the following data types:

  • xs:dateTime: The dateTime data type is used to specify a date and a time in the following form “YYYY-MM-DDThh:mm:ss.fffK” where:
    • YYYY indicates the year
    • MM indicates the month
    • DD indicates the day
    • T indicates the start of the required time section
    • hh indicates the hour
    • mm indicates the minute
    • ss indicates the second
    • fff indicates the milliseconds
    • K represents the time zone information of a date and time value (e.g. +05:00)
  • xs:date: The date data type is used to specify a date in the following form “YYYY-MM-DD” where:
    • YYYY indicates the year
    • MM indicates the month
    • DD indicates the day
  • xs:time: The time data type is used to specify a time in the following form “hh:mm:ss.fffK” where:
    • hh indicates the hour
    • mm indicates the minute
    • ss indicates the second
    • fff indicates the milliseconds
    • K represents the time zone information of a date and time value (e.g. +05:00)

or you could use an xs:string that b.asically accepts everything. The only problem here is that by default we can’t do a schema validation to see if it is a valid DateTime format.

But not all systems respect de DateTime formats expected by the XSD default values. So, what are my options if a system expects other types of DateTime, Date, or Time formats? Like:

  • MM/DD/YYYY
  • YYYY-DD-MM
  • YYYY-MM-DD HH:mm:ss
  • YYYYMMDD
  • HHmmss
  • HH:mm:ss
  • and so on.

Simple Type Derivation Using the Restriction Mechanism

Luckily for us BizTalk Schema Editor and schemas, in general, allow us to derive a simple type, for example, xs:string, by using the restriction mechanism, i.e., we are typically restricting the values allowed in a message for that attribute or element value to a subset of those values allowed by the base simple type. A good and common sample of these types of restrictions is to restrict a string type to be one of several enumerated strings.

Luckily for us, again, we can also apply a pattern (that uses Regex expression) to validate the element or attribute value.

To derive a simple type by using restriction:

  • Select the relevant Field Element node or Field Attribute node in the schema tree
  • And then, in the Properties window, on the Derived By property set as Restriction.
    • This will add/present the Restriction properties on the Properties window.
  • On the Restriction properties, click on the (3 dots) on the Pattern property to define the RegEx.

Regular expression samples to validate date formats

Here is where the fun starts. There are many ways to archive this goal:

  • One’s more simple but probably not that efficient since they may not validate all cases (Leap year, and so on)
  • Others more complex that requires more knowladge but more accurated.

In a general overview, the use of regex to validate the date format supports a variety of situations and possibilities like:

  • Rule to validate the year:
    • d{4} – it says that accepts 4 digits like: 2022
    • (19|20)[0-9][0-9] -accepts years starting with 19 or 20, i.e., from 1900 to 2099
  • Rule to validate the month:
    • d{2} – it says that accepts 2 digits like: 12, but the problem here is that also accepts invalid months like 24 or 99.
    • 0?[1-9]|1[012] – accepts 01-09 (leading zero), 1-9 (single digit) and 10,11,12
  • Rule to validate the day:
    • d{2} – it says that accepts 2 digits like: 12, but the problem here is that also accepts invalid days like 32 or 99. It also don’t validate what is the month we define to validate if accepts 28, 29, 30 or 31
    • 0?[1-9]|[12][0-9]|3[01] – accepts 01-09 (leading zero), 1-9 (single digit), 10-19, 20-29 and 30-31. It doesn.t check if it is a Leap year or not.
  • To implement the lead year that needs to be with a concatenation of several rules like this sample:
    • ^(?:(?:31(/)(?:0[13578]|1[02]))1|(?:(?:29|30)(/)(?:0[13-9]|1[0-2])2))(?:(?:18|19|20)d{2})$|^(?:29(/)023(?:(?:(?:(?:18|19|20))(?:0[48]|[2468][048]|[13579][26]))))$|^(?:0?[1-9]|1d|2[0-8])(/)(?:(?:0[1-9])|(?:1[0-2]))4(?:(?:18|19|20)d{2})$

This is a different approach to do the same as above:

  • (19|20)((([02468][48]|[13579][26])-0?2-29)|dd-((0?[469]|11)-([012]?d|30)|(0?[13578]|1[02])-([012]?d|3[01])|(0?2-([01]?d|2[0-8]))))

But we can go further and allow different types of format like:

  • ^(?:(?:31(/|-|.)(?:0?[13578]|1[02]|(?:Jan|Mar|May|Jul|Aug|Oct|Dec)))1|(?:(?:29|30)(/|-|.)(?:0?[1,3-9]|1[0-2]|(?:Jan|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec))2))(?:(?:1[6-9]|[2-9]d)?d{2})$|^(?:29(/|-|.)(?:0?2|(?:Feb))3(?:(?:(?:1[6-9]|[2-9]d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1d|2[0-8])(/|-|.)(?:(?:0?[1-9]|(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep))|(?:1[0-2]|(?:Oct|Nov|Dec)))4(?:(?:1[6-9]|[2-9]d)?d{2})$

Your imagination and skills are the limit.

Note: images and debug RegEx at https://www.debuggex.com/.

Using multiple patterns to simplify complexity

As you saw above, things can go out of control and become quite complex. Fortunately, the BizTalk Schema Editor and the schemas, in general, allow us to apply multiple patterns to simplify the overall expression.

So, for example, if I want to have the following Date format: YYYYMMDD with Leap year validated I can use the combination of these 4 expressions:

  • (19|20)dd(0[1-9]|1[0-2])(0[1-9]|1[0-9]|2[0-8])
  • (19|20)([02468][048]|[13579][26])0229
  • (19|20)dd(0[13-9]|1[0-2])(29|30)
  • (19|20)dd(0[13578]|1[02])31

Some samples

Date in the following format: YYYY-MM-DD like 2022-12-11

Simple formats

  • d{4}-d{2}-d{2} – simple format without validating month or day
  • (19|20)d{2}-d{2}-d{2} – restricting the year

Complex formats

  • (19|20)((([02468][48]|[13579][26])-0?2-29)|dd-((0?[469]|11)-([012]?d|30)|(0?[13578]|1[02])-([012]?d|3[01])|(0?2-([01]?d|2[0-8]))))

Date in the following format: YYYYMMDD like 20221211

Simple formats

  • d{4}d{2}d{2} – simple format without validating month or day
  • (19|20)d{2}d{2}d{2} – restricting the year

Complex formats

  • (19|20)dd(0[1-9]|1[0-2])(0[1-9]|1[0-9]|2[0-9]) – do not control Leap year

Date in the following format: MM/DD/YYYY like 12/11/1999

Simple formats

  • d{2}/d{2}/d{4} – simple format without validating month or day
  • d{2}/d{2}/(19|20)d{2} – restricting the year

Complex formats

  • ^(?:(?:31(/|-|.)(?:0?[13578]|1[02]))1|(?:(?:29|30)(/|-|.)(?:0?[1,3-9]|1[0-2])2))(?:(?:1[6-9]|[2-9]d)?d{2})$|^(?:29(/|-|.)0?23(?:(?:(?:1[6-9]|[2-9]d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1d|2[0-8])(/|-|.)(?:(?:0?[1-9])|(?:1[0-2]))4(?:(?:1[6-9]|[2-9]d)?d{2})$
  • ^([0]d|[1][0-2])/([0-2]d|[3][0-1])/([2][01]|[1][6-9])d{2}(s([0-1]d|[2][0-3])(:[0-5]d){1,2})?$

Date in the following format: YYYY-MM-DDZ like 2019-06-12Z

Simple formats

  • d{4}-d{2}-d{2}Z – simple format without validating month or day
  • (19|20)d{2}-d{2}-d{2}Z – restricting the year

Complex formats

  • ^dddd-(0?[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])Z?(-+:([0-5][0-9]))?$

Time in the following format: HH:mm:ss like 23:59:59

Simple formats

  • d{2}:d{2}:d{2} – simple format without validating valid hours, minutes or seconds

Complex formats

  • (([01][0-9]|2[0-3]):[0-5]:[0-9])

Time in the following format: HHmm like 2359

Simple formats

  • d{2}d{2} – simple format without validating month or day

Complex formats:

  • (([01][0-9]|2[0-3])[0-5])

Download

THIS SAMPLE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the POC: BizTalk Schemas Handle Restrictions on Date from GitHub here: