Checking for the existence of a property by Thomas Canter

Checking for the existence of a property by Thomas Canter

Today, I will bring back to life another old BizTalk Server blog post by an old friend of mine, Thomas Canter, with his permission, that I find pretty interesting and useful: Checking for the existence of a property. This was initially published on http://geekswithblogs.net/ThomasCanter, now retired.

NOTE:

  • The variable PropExists as bool has already been created
  • The Property of interest is BTS.RetryCount
  • The Message is Message_In

The list from Using Operators in Expressions (https://learn.microsoft.com/en-us/biztalk/core/using-operators-in-expressions) has the typical list of stuff that you expect in C#, multiplication, bit operations (shift left and right), and Boolean operators, but a couple of extremely useful constructs are available that are unique to BizTalk.

The most important of these (in my humble opinion) is the exists operator.

As you are all aware, to even check whether a property exists in an expression throws an exception… as in the following case:

PropExists = (Message_In(BTS.RetryCount) != null && Message_In(BTS.RetryCount) != “”);

If BTS.RetryCount does not exist in the message context, then the MissingPropertyException (Windows Event Log Event ID 10019) is thrown.

Without having to resort to a scope shape and exception handler, the exists operator allows you to check if a property exists in a message and is used in the following format:

PropExists = BTS.RetryCount exists Message_In;

OR

if (BTS.RetryCount exists Message_In)
{
     …;
}

Conclusion

Using the XLANG/s exists operator in your orchestration allows you to test for the existence of a property in a message without resorting to a scope shape and exception handler.

Below are a few more XLANG/s functions that can provide some value to your Orchestrations:

Operator Description Example
checked() raise error on arithmetic overflow checked(x = y * 1000)
unchecked() ignore arithmetic overflow unchecked(x = y * 1000)
succeeded() test for successful completion of transactional scope or orchestration succeeded()
exists test for the existence of a message context property BTS.RetryCount exists Message_In

Hope you find this helpful! So, if you liked the content or found it helpful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego! 

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

The maligned Distinguished Field or why Distinguished Fields are cool by Thomas Canter

The maligned Distinguished Field or why Distinguished Fields are cool by Thomas Canter

Today, I will bring back to life an old BizTalk Server blog post by an old friend of mine, Thomas Canter, with his permission, that I find pretty interesting and useful: The maligned Distinguished Field or why Distinguished Fields are cool.

The Distinguished Field is often seen as the weaker of the two types of Fields when handling Fields in BizTalk.

After all, the Distinguished Field can’t be used as a filter on a message, and it’s slower than its big brother the Promoted Field.

Well, today I’m here to dispel the myth of the wimpy Distinguished Field and place in the pantheon of power that equals, and in some ways exceeds the Promoted Field.

MYTH: Getting the value of a Distinguished Field requires loading the entire message into memory.

The first myth we need to dispel is that the Promoted Field is quicker to access than the Distinguished Field.

This is due to the statement in the old BizTalk Documentation (now deprecated but still possible to retrieve), and I quote:

One of the benefits of promoted Fields is that the value of the element that is promoted is available in the context of the message. This means that retrieving that value is inexpensive, as it does not require loading the message into memory to execute an XPath statement on the message.

from The BizTalk Server Message.

What is implied here is that for the Promoted Field reading, its value doesn’t require an XPath read into the message and, conversely, that the Distinguished Field does require loading the message and has a performance cost because it’s evaluated when queried.

Nothing could be further from the truth! In fact, both the Promoted and Distinguished Fields are evaluated at the same time, and both are placed in the message context at the same time. So, let’s talk about how fields get into the message context.

… Lots of stuff cut out…

Since Distinguished fields do not require a separate property schema, the evaluation of Distinguished fields by the Orchestration engine consumes less overhead than the evaluation of Property fields by the Orchestration engine. The evaluation of Property fields requires an XPath query, the evaluation of Distinguished fields does not require an XPath query as the pipeline disassembler populates the Distinguished fields in the context and the orchestration engine gets the cached values. However, if the orchestration engine does not find the property in the context, it will then evaluate the XPath query to find the value. Distinguished fields do not have a size limitation.

from About BizTalk Message Context Fields

Now, how does Promoted and Distinguished Fields get into the Message Context? This occurs automatically in the Receive Pipeline by specific pre-built pipeline components?

Out of the box, the BizTalk XML Disassembler, BizTalk Flat File Disassembler, and the BizTalk Framework Disassembler Promote Fields to the message context. All other production level Pipelines promote fields, most also support Distinguished Fields.

Distinguished Fields are written to the Message Context if one of these Receive Pipeline Components is used in the Pipeline. Interestingly enough, this explains why the Passthrough pipeline doesn’t promote Fields from the message. There are no components in the Passthrough Pipeline, it does nothing to the message content and therefore, nothing gets promoted, especially BTS.MessageType.

Regarding performance, Distinguished Fields beat out Promoted Fields 9 days each week. This is because both Promoted and Distinguished require the same overhead of writing the message value to the context Field bag in the Message Box. Still, Promoted Fields have the additional overhead of both being written to the Message Box context database AND the Subscription database. Promoted Fields have an impact every time a message is written to the Message Box because each Promoted Field that exists must be evaluated in a massive union (very efficiently written union, mind you!) that builds the list of matching activation subscriptions. So, in short, the more Promoted Fields that you have, the costlier the subscription process.

RECOMMENDATION: Use Promoted somewhat sparingly. Don’t avoid them, but do not use them if you do not need to. Use Promoted Fields as they were designed: to facilitate message routing, but not to make it easy to access a message value. Instead, primarily use Distinguished Fields.

MYTH: It’s always safe to use a Promoted or Distinguished Field in an Orchestration.

exists test for the existence of a message context property BTS.RetryCount exists Message_In

from Using Operators in Expressions

Let us talk about how to handle message content that is missing when it is a Promoted Field and a Distinguished Field. What we are talking about specifically is the field that was Promoted, or Distinguished did not exist in the inbound message. The XLANG/s xpath statement that was used to query the message for the content during pipeline processing returned a null object.

The first thing to understand is when a Promoted and Distinguished Field comes into existence. They are essentially the same, and this occurs when a Pipeline component parses a message and either Promotes or Writes the value to the context. The simple answer is that when the value does not exist, the Field is not created. A query to the context for the Field returns a null object.

So, if you attempt to access a Promoted or Distinguished Field that didn’t exist in the inbound message, you can cause an unhandled exception to be thrown. Specifically, in both cases, a NullReferenceException.

Promoted Fields have a special XLang/s test operator: exists (we will soon blog about this operator). You can use this operator to determine if they exist before attempting to use them. In this case, Promoted Fields can always be tested for existence before use and can safely be avoided when they don’t exist.

Unfortunately, Distinguished Fields don’t have such a special test operator, and can cause an unpreventable unhandled exception. Specifically, if you use a Field that the underlying type is a native non-nullable type. For instance, suppose the value that you have distinguished is an integer. Integers cannot be null (and yes, I am aware of the Nullable generics, but we are talking about what BizTalk XLANG/s has, not what is C# has), and if the underlying value didn’t exist, and you attempt to use the value, or even test to see if the value exists will cause an unhandled NullReferenceException when BizTalk’s XLang engine attempts to convert a null value into an integer by calling the System.Number.Parse(string) method with a null value.

Here comes in the kicker and why a Distinguished field can appear to be fine at design time but bite you at run-time.

At design time, the expression editor generates a pseudo-class-like dotted object for you to use in your expression. At run-time, there is simple type-casting that occurs by the run-time engine that inspects the XML datatype of the node in the Schema, retrieves the value as an object… then attempts to call the appropriate ConvertTo method on the Object. When casting a Null to an Int32 or any other intrinsic datatype, a NullReferenceException is thrown, and the Orchestration fails.

The primary difference (excluding Routing) between Promoted and Distinguished Fields is the developer’s design-time experience. Distinguished Fields are easy to use because they emulate .Net Class dotted notation.

RECOMMENDATION: If there is any chance that accessing the Distinguished Field may cause an exception, then place the check in a Scope Shape that has a catch shape to handle the NullReferenceException.

MYTH: Distinguished Fields are only accessible in Orchestrations

WRONG Documentation: Property Schemas
RIGHT Documentation: Distinguished Fields in Disassembler Pipeline ComponentsProcessing the MessagePromote Properties (Node Property of All Schemas)

Another major fallacy about Distinguished Fields is that they are only accessible in the Orchestration. This is also untrue. The BizTalk Server Documentation clearly has an example of how to use Distinguished Fields in any component from the RIGHT Documentation above.

All Distinguished Fields outside of an Orchestration use a fixed schema:
http://schemas.microsoft.com/BizTalk/2003/btsDistinguishedFields

The Field to use is the XPath of the node that is Distinguished, such as:

/*[local-name()='PO' and namespace-uri()='http://SendHtmlMessage.PO']/*[local-name()='Price' and namespace-uri()='']

Thus, to access this, you would use the Read Method:

MyObject = MyMessageContext.Read("/*[local-name()='PO' and namespace-uri()='http://SendHtmlMessage.PO']/*[local-name()='Price' and namespace-uri()='']", " http://schemas.microsoft.com/BizTalk/2003/btsDistinguishedFields”);

If the Field exists, then MyObject will contain an object that can be cast to the appropriate type.

RECOMMENDATION: Once the proper Pipeline Component has processed the message, use the Distinguished Field as you would any Field without the Xpath lookup overhead.

MYTH: Distinguished Fields in Orchestration Expression shapes are actually code.

You have to hand it to the people who did the coding for XLANG/s. It looks like C#, it feels like C#, and 99% of the time, it pretty much generates standard C#.

In many ways, this is not your father’s C#. It is really XLANG/s, and it has its own syntax and special components. Distinguished Fields are a prime example.

Think back on all the times you used a distinguished Field. It feels like it’s a C# Object! It uses dotted notation (Node.Node.Node.Attribute). You assign values to it, you use its value in an expression, and it comes out as the correct type. When the node is Boolean, then it behaves like a Boolean. Nothing could be further from the actual behavior. Just because it looks like a duck, doesn’t mean that it’s a duck. It really is a trick, that the Expression Editor parses the XSD on the fly and generates a classlike editor experience, but no actual code ever gets generated.

Hope you find this helpful! So, if you liked the content or found it helpful and want to help me write more content, you can buy (or help buy) my son a Star Wars Lego! 

Author: Sandro Pereira

Sandro Pereira lives in Portugal and works as a consultant at DevScope. In the past years, he has been working on implementing Integration scenarios both on-premises and cloud for various clients, each with different scenarios from a technical point of view, size, and criticality, using Microsoft Azure, Microsoft BizTalk Server and different technologies like AS2, EDI, RosettaNet, SAP, TIBCO etc.

He is a regular blogger, international speaker, and technical reviewer of several BizTalk books all focused on Integration. He is also the author of the book “BizTalk Mapping Patterns & Best Practices”. He has been awarded MVP since 2011 for his contributions to the integration community.
View all posts by Sandro Pereira

Best Practices while working with Schemas Pattern restrictions

Best Practices while working with Schemas Pattern restrictions

Yesterday I spoke about how you can apply custom pattern restrictions to properly validate DateTime, Date, and Time formats inside elements or attributes. You can see more about it here:

And I mention that these Regular Expressions can be simple and relatively easier to read if you have some basic knowledge like this one below:

  • d{4}d{2}d{2} – that is a simple format that expects 4 digits followed by 2 additional digits and another 2 digits that is the date in the YYYYMMDD format without validating the accuracy of months or days and where:
    • YYYY is the for digits year, like 2022.
    • MM is the 2 digits month, like 01.
    • DD is the 2 digits day, like 21.

But it can get highly complex that even people with good knowledge have difficulty translating the expression to the expected pattern, like the one below:

  • ^(?:(?:31(/|-|.)(?:0?[13578]|1[02]))1|(?:(?:29|30)(/|-|.)(?:0?[1,3-9]|1[0-2])2))(?:(?:1[6-9]|[2-9]d)?d{2})$|^(?:29(/|-|.)0?23(?:(?:(?:1[6-9]|[2-9]d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1d|2[0-8])(/|-|.)(?:(?:0?[1-9])|(?:1[0-2]))4(?:(?:1[6-9]|[2-9]d)?d{2})$

It is not simple to look to this RegEx and say: “yep, we are expecting this Date format: MM/DD/YYYY like 12/11/1999, and by the way, it validates the Leap Year!”

Imagine users that don’t have strong know-how about RegEx!

So, what can we do to improve this experience? What are the best practices in these cases?

Best practices

A good best practice to improve readability while documenting your schemas is to add notes to these elements or attributes.

We can and should use the Notes property to enter notes, such as comments related to the business process, that you would like to make about the selected RecordField Element, or Field Attribute node. In these DateTime, Date, and Time cases, we can simply add the expected format in the Notes property, like:

  • Format: YYYY-MM-DD
  • Format: HH:mm:ss
  • Format: YYYYMMDD
  • Format: HHmm
  • Format: YYYY-MM-DD – It validates Leap Year

To accomplish this, we need to

  • Right-click on the Element or Attribute fields in the schema tree view and select the Properties option.
  • On the Properties window, click on the … (three dots) on the Notes property. This action will open a Notes window where you can add all your relevant notes.

Do that to all your elements and fields that are using pattern restrictions.

Even non-technical guys can understand the Schema specification you provide or are consuming. This best practice implementation will also help you improve productivity since you will not spend too much time decompiling Regular Expressions.

Download

THIS SAMPLE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the POC: BizTalk Schemas Handle Restrictions on Date from GitHub here:

BizTalk Schema Validation: DateTime Restrictions

BizTalk Schema Validation: DateTime Restrictions

Today I was involved in a BizTalk Schema importation that includes not-so-used restrictions on Date and Time elements formats. And that gave me the idea and inspiration to create this blog post.

When we work with DateTime on Schemas by default we can choose from the following data types:

  • xs:dateTime: The dateTime data type is used to specify a date and a time in the following form “YYYY-MM-DDThh:mm:ss.fffK” where:
    • YYYY indicates the year
    • MM indicates the month
    • DD indicates the day
    • T indicates the start of the required time section
    • hh indicates the hour
    • mm indicates the minute
    • ss indicates the second
    • fff indicates the milliseconds
    • K represents the time zone information of a date and time value (e.g. +05:00)
  • xs:date: The date data type is used to specify a date in the following form “YYYY-MM-DD” where:
    • YYYY indicates the year
    • MM indicates the month
    • DD indicates the day
  • xs:time: The time data type is used to specify a time in the following form “hh:mm:ss.fffK” where:
    • hh indicates the hour
    • mm indicates the minute
    • ss indicates the second
    • fff indicates the milliseconds
    • K represents the time zone information of a date and time value (e.g. +05:00)

or you could use an xs:string that b.asically accepts everything. The only problem here is that by default we can’t do a schema validation to see if it is a valid DateTime format.

But not all systems respect de DateTime formats expected by the XSD default values. So, what are my options if a system expects other types of DateTime, Date, or Time formats? Like:

  • MM/DD/YYYY
  • YYYY-DD-MM
  • YYYY-MM-DD HH:mm:ss
  • YYYYMMDD
  • HHmmss
  • HH:mm:ss
  • and so on.

Simple Type Derivation Using the Restriction Mechanism

Luckily for us BizTalk Schema Editor and schemas, in general, allow us to derive a simple type, for example, xs:string, by using the restriction mechanism, i.e., we are typically restricting the values allowed in a message for that attribute or element value to a subset of those values allowed by the base simple type. A good and common sample of these types of restrictions is to restrict a string type to be one of several enumerated strings.

Luckily for us, again, we can also apply a pattern (that uses Regex expression) to validate the element or attribute value.

To derive a simple type by using restriction:

  • Select the relevant Field Element node or Field Attribute node in the schema tree
  • And then, in the Properties window, on the Derived By property set as Restriction.
    • This will add/present the Restriction properties on the Properties window.
  • On the Restriction properties, click on the (3 dots) on the Pattern property to define the RegEx.

Regular expression samples to validate date formats

Here is where the fun starts. There are many ways to archive this goal:

  • One’s more simple but probably not that efficient since they may not validate all cases (Leap year, and so on)
  • Others more complex that requires more knowladge but more accurated.

In a general overview, the use of regex to validate the date format supports a variety of situations and possibilities like:

  • Rule to validate the year:
    • d{4} – it says that accepts 4 digits like: 2022
    • (19|20)[0-9][0-9] -accepts years starting with 19 or 20, i.e., from 1900 to 2099
  • Rule to validate the month:
    • d{2} – it says that accepts 2 digits like: 12, but the problem here is that also accepts invalid months like 24 or 99.
    • 0?[1-9]|1[012] – accepts 01-09 (leading zero), 1-9 (single digit) and 10,11,12
  • Rule to validate the day:
    • d{2} – it says that accepts 2 digits like: 12, but the problem here is that also accepts invalid days like 32 or 99. It also don’t validate what is the month we define to validate if accepts 28, 29, 30 or 31
    • 0?[1-9]|[12][0-9]|3[01] – accepts 01-09 (leading zero), 1-9 (single digit), 10-19, 20-29 and 30-31. It doesn.t check if it is a Leap year or not.
  • To implement the lead year that needs to be with a concatenation of several rules like this sample:
    • ^(?:(?:31(/)(?:0[13578]|1[02]))1|(?:(?:29|30)(/)(?:0[13-9]|1[0-2])2))(?:(?:18|19|20)d{2})$|^(?:29(/)023(?:(?:(?:(?:18|19|20))(?:0[48]|[2468][048]|[13579][26]))))$|^(?:0?[1-9]|1d|2[0-8])(/)(?:(?:0[1-9])|(?:1[0-2]))4(?:(?:18|19|20)d{2})$

This is a different approach to do the same as above:

  • (19|20)((([02468][48]|[13579][26])-0?2-29)|dd-((0?[469]|11)-([012]?d|30)|(0?[13578]|1[02])-([012]?d|3[01])|(0?2-([01]?d|2[0-8]))))

But we can go further and allow different types of format like:

  • ^(?:(?:31(/|-|.)(?:0?[13578]|1[02]|(?:Jan|Mar|May|Jul|Aug|Oct|Dec)))1|(?:(?:29|30)(/|-|.)(?:0?[1,3-9]|1[0-2]|(?:Jan|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec))2))(?:(?:1[6-9]|[2-9]d)?d{2})$|^(?:29(/|-|.)(?:0?2|(?:Feb))3(?:(?:(?:1[6-9]|[2-9]d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1d|2[0-8])(/|-|.)(?:(?:0?[1-9]|(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep))|(?:1[0-2]|(?:Oct|Nov|Dec)))4(?:(?:1[6-9]|[2-9]d)?d{2})$

Your imagination and skills are the limit.

Note: images and debug RegEx at https://www.debuggex.com/.

Using multiple patterns to simplify complexity

As you saw above, things can go out of control and become quite complex. Fortunately, the BizTalk Schema Editor and the schemas, in general, allow us to apply multiple patterns to simplify the overall expression.

So, for example, if I want to have the following Date format: YYYYMMDD with Leap year validated I can use the combination of these 4 expressions:

  • (19|20)dd(0[1-9]|1[0-2])(0[1-9]|1[0-9]|2[0-8])
  • (19|20)([02468][048]|[13579][26])0229
  • (19|20)dd(0[13-9]|1[0-2])(29|30)
  • (19|20)dd(0[13578]|1[02])31

Some samples

Date in the following format: YYYY-MM-DD like 2022-12-11

Simple formats

  • d{4}-d{2}-d{2} – simple format without validating month or day
  • (19|20)d{2}-d{2}-d{2} – restricting the year

Complex formats

  • (19|20)((([02468][48]|[13579][26])-0?2-29)|dd-((0?[469]|11)-([012]?d|30)|(0?[13578]|1[02])-([012]?d|3[01])|(0?2-([01]?d|2[0-8]))))

Date in the following format: YYYYMMDD like 20221211

Simple formats

  • d{4}d{2}d{2} – simple format without validating month or day
  • (19|20)d{2}d{2}d{2} – restricting the year

Complex formats

  • (19|20)dd(0[1-9]|1[0-2])(0[1-9]|1[0-9]|2[0-9]) – do not control Leap year

Date in the following format: MM/DD/YYYY like 12/11/1999

Simple formats

  • d{2}/d{2}/d{4} – simple format without validating month or day
  • d{2}/d{2}/(19|20)d{2} – restricting the year

Complex formats

  • ^(?:(?:31(/|-|.)(?:0?[13578]|1[02]))1|(?:(?:29|30)(/|-|.)(?:0?[1,3-9]|1[0-2])2))(?:(?:1[6-9]|[2-9]d)?d{2})$|^(?:29(/|-|.)0?23(?:(?:(?:1[6-9]|[2-9]d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1d|2[0-8])(/|-|.)(?:(?:0?[1-9])|(?:1[0-2]))4(?:(?:1[6-9]|[2-9]d)?d{2})$
  • ^([0]d|[1][0-2])/([0-2]d|[3][0-1])/([2][01]|[1][6-9])d{2}(s([0-1]d|[2][0-3])(:[0-5]d){1,2})?$

Date in the following format: YYYY-MM-DDZ like 2019-06-12Z

Simple formats

  • d{4}-d{2}-d{2}Z – simple format without validating month or day
  • (19|20)d{2}-d{2}-d{2}Z – restricting the year

Complex formats

  • ^dddd-(0?[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])Z?(-+:([0-5][0-9]))?$

Time in the following format: HH:mm:ss like 23:59:59

Simple formats

  • d{2}:d{2}:d{2} – simple format without validating valid hours, minutes or seconds

Complex formats

  • (([01][0-9]|2[0-3]):[0-5]:[0-9])

Time in the following format: HHmm like 2359

Simple formats

  • d{2}d{2} – simple format without validating month or day

Complex formats:

  • (([01][0-9]|2[0-3])[0-5])

Download

THIS SAMPLE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

You can download the POC: BizTalk Schemas Handle Restrictions on Date from GitHub here:

BizTalk Server Schema Editor getting blocked by Internet Explorer Enhanced Security Configuration

BizTalk Server Schema Editor getting blocked by Internet Explorer Enhanced Security Configuration

I have been working on a new BizTalk Server project on a new client for a few weeks. Now that we are beginning the development phase, I was getting annoying with an Internet Explorer security blocking windows every time I try to open a schema document on Visual Studio with the BizTalk Server Schema Editor:

Internet Explorer

Content within this application coming from the website listed below is being blocked by Internet Explorer Enhanced Security Configuration:

about:security_devenv.exe

To be honest, I initially ignored this issue and immediately clicked on the Close button, and everything would work as usual. But starting to work every day on the project and facing this issue every time I tried to open a Schema was simple to annoying.

Cause

We don’t need the IE to develop BizTalk Schemas, but the XSD viewer, in fact, opens an IE embedded inside Visual Studio. And that is the reason for this issue.

I typically don’t get this “warning” message because I usually turn off Internet Explorer Enhanced Security Configuration on my BizTalk Servers.

Solution

The solution is simple:

  • You can turn off Internet Explorer Enhanced Security Configuration by:
    • Start by running the Server Manager, if it is not already open, from either:
      • On the Windows taskbar, click the Server Manager button
      • On the Start screen, click Server Manager
    • In the Server Manager Dashboard, from the scope pane (on the left side) click on Local Server
    • In the Server Properties for the Local Server, you’ll see the option for IE Enhanced Security Configuration. Click On to change the option
    • A dialog box appears, letting Internet Explorer Enhanced Security Configuration be enabled/disabled separately for normal users and administrators; turn off both. After disabling both options, click OK
    • Click the Refresh button at the top of the Server Manager and the IE Enhanced Security Configuration should now show as Off
  • Or, if don’t want to turn off Internet Explorer Enhanced Security Configuration , you can on the blocking pop-up window click on Add
    • On the Trusted sites window, make sure Require server verification (https:) for all sites in this zone is not marked and then click Add

After one of these two options/settings that annoying blocking behavior inside Visual Studio will be gone.

The post BizTalk Server Schema Editor getting blocked by Internet Explorer Enhanced Security Configuration appeared first on SANDRO PEREIRA BIZTALK BLOG.

BizTalk Property Schema Error: The type name ‘…’ does not exist in the type ‘System’

BizTalk Property Schema Error: The type name ‘…’ does not exist in the type ‘System’

I’m just playing with BizTalk Server doing a small proof-of-concept using BizTalk Server and Azure Service Bus, and I was surprised with a few bunches of errors while I tried to compile my simple project:

Severity Code Description Project File Line Suppression State
Error CS0426 The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’ POC.BizTalk.AzureServiceBus C:DEVPOCPOC.BizTalk.AzureServiceBusPOC.BizTalk.AzureServiceBusSchemasASBPropertySchema.xsd.cs 9 Active
Severity Code Description Project File Line Suppression State
Error CS0426 The type name ‘Xml’ does not exist in the type ‘System’ POC.BizTalk.AzureServiceBus C:DEVPOCPOC.BizTalk.AzureServiceBusPOC.BizTalk.AzureServiceBusSchemasASBPropertySchema.xsd.cs 85 Active

Were is the full error list:

  • The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘SerializableAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
  • The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘SerializableAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
  • The type name ‘Xml’ does not exist in the type ‘System’
  • The type name ‘Xml’ does not exist in the type ‘System’
  • The type name ‘Type’ does not exist in the type ‘System’
  • The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘SerializableAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
  • The type name ‘Xml’ does not exist in the type ‘System’
  • The type name ‘Xml’ does not exist in the type ‘System’
  • The type name ‘Type’ does not exist in the type ‘System’
  • The type name ‘SerializableAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘SerializableAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttributeAttribute’ does not exist in the type ‘System’
  • The type name ‘NonSerializedAttribute’ does not exist in the type ‘System’
  • The type name ‘Xml’ does not exist in the type ‘System’
  • The type name ‘Xml’ does not exist in the type ‘System’
  • The type name ‘Type’ does not exist in the type ‘System’

Cause

Initially, I have to be honest, I was not realizing why this error was happening, mainly because the main error description may elude us and point us to DLL reference problems: Xml, System, SerializableAttributeAttribute, and so on. But looking carefully to the error message details, all of the errors will point us to the PropertySchema file.

After realizing that, it was not difficult to realize that I had on my Property Schema an element call System. System is a reserved word that you CANNOT use inside the property schemas.

Solution

The solution to this issue is quite simple, you need to:

  • Rename your System element inside your Property Schema to another word, for example ExtSystem
  • Of course, fix all the dependencias inside your project, if you were already using this element iside your solution, for example Message Assigment shape
  • and finally compile the project

The post BizTalk Property Schema Error: The type name ‘…’ does not exist in the type ‘System’ appeared first on SANDRO PEREIRA BIZTALK BLOG.

BizTalk Server tips and tricks for developers: How to create a .NET class from a schema

BizTalk Server tips and tricks for developers: How to create a .NET class from a schema

Sometimes we want to bypass the adapters and perform the communication thru .NET code. Sometimes we want to access multiple values, or recursive values, of a message inside a helper class that supports an orchestration. Sometimes we want to perform a complex transformation or construct a message thru .NET code instead of a BizTalk Server map.

And for each of these circumstances, we can have several approaches, one that is simple, effective, and transversal to all of them is to use the XML Schema to generate a .NET class. This way, you have a quick and straightforward way to represent the messages you get from your BizTalk processes into a .NET class.

The easy way to generate classes that conform to a specific schema is to use the need to use the XML Schema Definition tool (Xsd.exe). You can do that by:

  • Open a Developer Command Prompt for VS <edition>
  • On the command prompt, navigate to the Schema folder and type
    • xsd /classes /language:CS Schema1.xsd

For complex schemas that import or include other schemas you need to provide all the dependencies, for example:

  • xsd /classes /language:CS Schema1.xsd Schema2.xsd

Now you will be able to import the C# class generated by the tool to your helper project and the rest is simple!

On the Orchestration you can create a variable that represents that class:

  • On property Type select <.NET Class…> and select the class that you created previously
  • From the Browse and Select .Net Type to reference, you need to select the proper assembly and the Type.

Then by using C# code inside Message Assign or Expression Shape you can serialize or deserialize XML document into C# and vice versa in a simple and straightforward way

//CONVERT MESSAGE INTO C# OBJECT
varPersonMsg = msgInput;

//CONVERT C# OBJECT INTO BIZTALK MESSAGE
msgOutput4 = varPersonMsg;

or if we have a function in a helper class like:

public static XmlDocument MapPersons(Person person)
{
  ...
}

we can directly pass the message to the function without the need to create a variable:

msgOutput4 = Support.Mapping.MapPersons(msgInput);

The post BizTalk Server tips and tricks for developers: How to create a .NET class from a schema appeared first on SANDRO PEREIRA BIZTALK BLOG.

Visual Studio BizTalk Schema Generator Wizard error: Error occurred while creating the BizTalk port configuration file

Visual Studio BizTalk Schema Generator Wizard error: Error occurred while creating the BizTalk port configuration file

After several days migrating BizTalk open-source contributions like BizTalk Mapper Extensions UtilityPack, BizTalk Pipeline Components Extensions Utility Pack, BizTalk MapperExtensions Functoid Project Template, BizTalk MapperExtensions Functoid Wizard and several tools like BizTalk Filter Finder Tool, BizTalk Bindings Exporter Tool, or BizTalk Port Multiplier Tool for BizTalk Server 2020. It is time to get back to one of my favorite topics error and warnings, cause, and solutions blog post. This time on a small development issue that I got recently while I was trying to Consume Adapter Service to generate the Oracle schemas inside Visual Studio.

I have done these thousands of times and it is a very straightforward task once you know how to communicate with Oracle system but this time I got the following error:

Error occurred while creating the BizTalk port configuration file. Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index

Consume Adapter Service Schema Generator BizTalk port configuration error

A curiosity is that the Wizard was able to successfully generate the Oracle Schemas. The problem occurred while it was trying to generate the Binding file.

Cause

Unfortunately, I don’t know exactly the cause reason for this error. In my view, the same occurred due to some special character coming from the Oracle resources that are being consumed or incompatibilities between Oracle data types and .NET data types and that are used to generate the Binding file.

Nevertheless, this is not a stopping issue. You still have all the necessary BizTalk resources generated by the wizard: the Oracle schemas. The only thing that is not generated is the binding file, which is extremely useful to create the receive or send port in the BizTalk Server Administration Console. However, despite this constraint, you are still able to manually create the port without requiring the binding file.

Solution

Well, you know me, it is possible to manually create the ports without requiring the binding files, however, this is an accelerator that I prefer not to lose. So, I had to investigate and solve this problem, before it appears more often.

And in fact, the solution is quite easy:

  • On the Consume Adapter Service Schema Generator Wizard, while you are configuring the connection string to Oracle, configure the URI, select the Binding Properties tab.
Consume Adapter Service Schema Generator BizTalk port configuration solution
  • On the Binding Properties tab, scroll down to the Metadata section. There you will find a property called: EnableSafeTyping.
    • This feature controls how the adapter surfaces certain Oracle data types and by default this value is false.
  • To solve this issue you need to change the EnableSafeTyping value to true.
Consume Adapter Service Schema Generator BizTalk port configuration solution

Since not all .NET and Oracle types are created equally, we occasionally need to set the value true for this property to handle some constraints around data types.

This is not a unique Oracle issue, this same error may happen when you are trying to generate schemas from SAP also.

The post Visual Studio BizTalk Schema Generator Wizard error: Error occurred while creating the BizTalk port configuration file appeared first on SANDRO PEREIRA BIZTALK BLOG.

BizTalk JSON Schema Wizard: Error in JSON Instance File.XMLNodeConverter can only convert JSON that begins with an object

BizTalk JSON Schema Wizard: Error in JSON Instance File.XMLNodeConverter can only convert JSON that begins with an object

This week while trying to use the BizTalk Server 2016 JSON Schema Wizard to generate an XML schema from a specific JSON file to be sent to a Logic App I got the following error:

Error in JSON Instance File.XMLNodeConverter can only convert JSON that begins with an object. Path ‘’, line 1, position 1.

The JSON file had the following format:

[
  {
    "IntID": 208,
    "ItemLogID": 14255826,
    "Step": "IN",
    "BusinessUnit": "TST",
  },
  {
    "IntID": 209,
    "ItemLogID": 14257419,
    "Step": "IN",
    "BusinessUnit": "TST",
  }
]

Basically, I’m trying to send a list of “objects” that in my case are Locks, to be processed.

Cause

The cause of this “problem” is that a JSON array of objects, it may not make sense in a BizTalk Server XML world as Morten la Cour very well written in this forum. At that sentence, I will add: “at it his being done today”. Because with a few improvements could be smarter and support this type of messages.

Why? The BizTalk
JSON Schema Wizard is a simple and “stupid” converter, it will ask only for you
to provide a root node and a namespace that it will add to the XML Schema and
the XML JSON representation because it will require that information to be able
to uniquely identify this type of message.

But it will not understand what is a “not identified” object array because it needs to give it a Record name in the XML equivalent.

So, that means
that it will not support JSON arrays?

No, it will support JSON arrays if you provide a field name to that array, i.e., instead of having:

[
…
]

You should have:

{
   "field name": [
   …
   ]
}

Solution

If you don’t have the control over that JSON message,
you may need to create a custom pipeline component to add or remove this field
name that will identify the array in the XML equivalent message.

If you have control over the structure of the JSON
message the simple way is to modify the structure of the message to include a
field name to identify the array. Lucky it was my case, so I modify the
original structure descrived above to be:

  {
    "locks": [
    {
      "IntID": 208,
      "ItemLogID": 14255826,
      "Step": "IN",
      "BusinessUnit": "TST",
    },
    {
      "IntID": 209,
      "ItemLogID": 14257419,
      "Step": "IN",
      "BusinessUnit": "TST",
    }
  ]
}

Now, if I try to run the BizTalk JSON Schema Wizard
against this message it will be able to create the JSON Schema.

The post BizTalk JSON Schema Wizard: Error in JSON Instance File.XMLNodeConverter can only convert JSON that begins with an object appeared first on SANDRO PEREIRA BIZTALK BLOG.