I've previously posted my ideas about using LINQ in BizTalk . At that point it was just theories, and following the links in the comments section of that post I wasn't the first to have them. Thanks to the red bits/green bits approach BizTalk…(read more)
Richard Seroter hasfinished writing his articles on TopXML.com about BizTalk and WCF. He goes into a lot of detail on all that can be done with the WCF adapters in BizTalk and also the WCF Adapter Pack for Line of Business Applications. If you work with BizTalk and are interested in WCF the articlesare a […]
Richard Seroter hasfinished writing his articles on TopXML.com about BizTalk and WCF. He goes into a lot of detail on all that can be done with the WCF adapters in BizTalk and also the WCF Adapter Pack for Line of Business Applications. If you work with BizTalk and are interested in WCF the articlesare a […]
Earlier this week, I grabbed a couple new CDs, hopped in the car, and drove down to San Diego to present at the .NET User Group’s Connected Systems meeting. The topic was the BizTalk Adapter Pack and I outlined what the BAP is, what the core use cases are, and demonstrated how to build a […]
Earlier this week, I grabbed a couple new CDs, hopped in the car, and drove down to San Diego to present at the .NET User Group’s Connected Systems meeting. The topic was the BizTalk Adapter Pack and I outlined what the BAP is, what the core use cases are, and demonstrated how to build a […]
Recently I have gotten a rare chance to step away from writing word documents and working with Visio to get to do some exciting BizTalk testing.
As part of my testing, I was pushing BizTalk and various Adapters until they failed. In doing so, sometimes this caused some odd side effects.
Recently, I was seeing thousands of Active Service Instances in BizTalk Administration that looked like this:
The Service Name was simply “Name is not available”. To make matters worse, I was not able to terminate the instances. I tried stopping and restarting the hosts, restarting the applications, restarting the Server, but nothing seemed to work.
Out of pure luck, I decided to check the SQL Agent Jobs… and sure enough the SQL Agent was not running. During all my performance testing none of the Message Box clean-up jobs ran and that left BizTalk in an undesirable state. I started the SQL Agent, gave the jobs a few minutes to run, and then was able to clean out the Active service instances.
So the moral of the story, if you see odd BizTalk behavior make sure you check your SQL Agent to ensure it is running and the jobs are completing successfully.
Another minor item I ran into deals with the Commerce Server Catalog Adapter. It has a property called “Import Directory”. The Adapter uses this as a temporary file store when submitting messages. This means the BizTalk Host will need read /write access to this location. This is something that can be easily forgotten… like I did.
So what happens if you forget to update this property or to give the folder the correct permissions?
Bad Things. And in BizTalk world, bad things translates into 4 Error Messages per input message in the event log.
Why we get so many errors – I do not know. If I remember correctly, none of them were very useful in correcting the problem.
As most of you know, during the day I’m a mild-mannered .NET developer using all things Microsoft. But I lead a double life!
Before my (still undiagnosed) illness came upon me suddenly last year, most of my free time was spent with my family or playing golf at some of the country's best daily-fee courses here in Houston. I really enjoyed the fresh air, exercise and competitive nature of the sport. Unfortunately, my current condition makes playing golf almost impossible and much too risky. This situation has left me with much more free time than I'm normally used to having and after several months of rest and recuperation, I've decided to do some "after-hours" consulting.
Now in the evenings and on the weekends I do freelance development work using mostly “other technologies” such as Javascript, Ajax, PHP and mySQL on OS X Leopard. I also do some freelance photography in my local area and occasionally work on native and web iPhone applications. Since this part-time activity has begun to generate income, I've also started a new part-time company called Jeff Lynch Development, Ltd. and launched a new blog at http://blog.jefflynchdev.com (wordpress.com) to post about Mac & iPhone development, my new business and life in general.
I still plan to post regularly on this blog about all things Microsoft, so don't fret! But if you're like many of us and have a foot on both sides of the Microsoft and Apple fence, please join me at http://blog.jefflynchdev.com and have a good read! I'll also let you know when my new web site is up and running!
Currently reading: Inside Steve's Brain by Leander Kahney
As most of you know, during the day I’m a mild-mannered .NET developer using all things Microsoft. But I lead a double life!
Before my (still undiagnosed) illness came upon me suddenly last year, most of my free time was spent with my family or playing golf at some of the country's best daily-fee courses here in Houston. I really enjoyed the fresh air, exercise and competitive nature of the sport. Unfortunately, my current condition makes playing golf almost impossible and much too risky. This situation has left me with much more free time than I'm normally used to having and after several months of rest and recuperation, I've decided to do some "after-hours" consulting.
Now in the evenings and on the weekends I do freelance development work using mostly “other technologies” such as Javascript, Ajax, PHP and mySQL on OS X Leopard. I also do some freelance photography in my local area and occasionally work on native and web iPhone applications. Since this part-time activity has begun to generate income, I've also started a new part-time company called Jeff Lynch Development, Ltd. and launched a new blog at http://blog.jefflynchdev.com (wordpress.com) to post about Mac & iPhone development, my new business and life in general.
I still plan to post regularly on this blog about all things Microsoft, so don't fret! But if you're like many of us and have a foot on both sides of the Microsoft and Apple fence, please join me at http://blog.jefflynchdev.com and have a good read! I'll also let you know when my new web site is up and running!
Currently reading: Inside Steve's Brain by Leander Kahney
The ability to rigidly define the content of a data field is an essential element in any structured data definition. It is no surprise therefore one can define data constraints on elements and attributes in a schema through the means of various facets, such as enumerations, min length, regular expressions, and many more. The implementation of facets in a schema provides a large amount of value in terms of the data aspect of the contract to be used in an integration. Whether the integration is WCF-based or not, this data contract must be adhered to and messages that do not adhere to this contract must not be processed.
There is, however, no value in defining the data constraints if these constraints can not be enforced at the time the data is received, as allowing "invalid" data to be processed will invariably lead to errors in the processing of that data. It is for this reason that validating a message against its schema is oftentimes a necessary requirement. Some might say that this validation should be performed by default, and any messages that do not validate should be rejected as early in the processing of the message as possible. BizTalk does not do this by default.
At this point you may be saying what do you mean "BizTalk does not validate my message?" … hence, the title of this post.
The point is that BizTalk CAN validate your message, but it does not do so by default. One of the reasons for this is flexibility: you may prefer to do the validation later in your process, perhaps in an orchestration, where you can handle the validation errors in your own way; or you may want the receive pipeline to do the validation and have BizTalk route the failed message. This decision is not made for you by BizTalk … your solution design will determine the appropriate implementation.
In this post I will present two ways in which you can get BizTalk to fully validate your message against its schema as early in the receiving process as possible: at the receive pipeline.
When using the standard XMLReceive pipeline in a receive location you can click on the ellipsis button to the right of the pipeline selection box to display the "Configure Pipeline" window. The screenshot below shows how the XMLReceive pipeline makes use of the XML disassembler pipeline component and the Party resolution pipeline component. When first setting up the receive location the ValidateDocument and DocumentSpecNames properties are False and blank respectively. In this default configuration the XMLReceive pipeline will try and match the received XML document’s target namespace and root node name to the published schemas in BizTalk, and the received document’s structure will be validated against the schema’s structure. No data validation is performed on the received document, however.
By setting the ValidateDocument property to True the XML disassembler pipeline component will be instructed to validate the data contained in the received document as well. Setting this property to true also requires that at least one DocumentSpecName is also provided, as the XML disassembler uses the schema specified by the fully qualified assembly name in the DocumentSpecNames property to identify the schema with which to perform the validation. With these properties set, if the receive location processes a message that fails validation an entry like the one below will be logged to the event log.
While this approach achieves the objective it falls short for an ideal solution, because:
This leads on to the second method to ensure that BizTalk validates the content of the received message: a custom pipeline that can be used in any receive location to do full schema validation, without requiring any additional configuration.
The concept behind the XMLValidatingReceive pipeline is to have a generic pipeline that one can use in any project, and in any receive location where the received document’s structure and data content needs to be validated against an existing schema. As the standard functionality of matching a received document with a deployed schema is still required, the XMLReceive pipeline is an excellent starting point. As the XMLReceive pipeline uses the XML disassembler and Party resolution pipeline components, the XMLValidatingReceive pipeline will need to use these two pipeline components as a starting point as well. In addition to these components, the XMLValidatingReceive pipeline will also make use if the XML validator pipeline component.
To create the XMLValidatingReceive pipeline, create a new receive pipeline. I created this pipeline in a generic BizTalk solution and project, so that I could use it for any solution, at any customer. In the new receive pipeline, add the XML disassembler, XML validator and Party resolution pipeline components, as per the screenshot below.
There is no need to change any of the default properties for any of these components. The pipeline can now be deployed and used in any BizTalk application. A message processed by this pipeline will now use the XML disassembler’s schema resolution functionality and it will validate the structure of the document against the resolved schema. The XML validator component will then use the resolved schema name to validate the received document’s data content, in accordance with any facets defined in the schema. Where a received document fails validation, the same error as in the previous method is shown (as below), except that the pipeline referred to will now be the new custom pipeline.
The result is a common pipeline that can be used in any receive location, and which does not require any configuration of pipeline properties to ensure that the received document is full validated. At the same time, this pipeline does not require any changes to be effected in the event of a change to a schema.
In conclusion, don’t assume that BizTalk will enforce those facets you diligently applied to your schema. You need to instruct BizTalk to validate a received message against the data constraints you define in your schema, and there are two simple ways in which this can be achieved. If it is a regular requirement to ensure that the received document is fully validated against the schema, then create a custom XMLValidatingReceive pipeline that can be used in any BizTalk development project.
The ability to rigidly define the content of a data field is an essential element in any structured data definition. It is no surprise therefore one can define data constraints on elements and attributes in a schema through the means of various facets, such as enumerations, min length, regular expressions, and many more. The implementation of facets in a schema provides a large amount of value in terms of the data aspect of the contract to be used in an integration. Whether the integration is WCF-based or not, this data contract must be adhered to and messages that do not adhere to this contract must not be processed.
There is, however, no value in defining the data constraints if these constraints can not be enforced at the time the data is received, as allowing "invalid" data to be processed will invariably lead to errors in the processing of that data. It is for this reason that validating a message against its schema is oftentimes a necessary requirement. Some might say that this validation should be performed by default, and any messages that do not validate should be rejected as early in the processing of the message as possible. BizTalk does not do this by default.
At this point you may be saying what do you mean "BizTalk does not validate my message?" … hence, the title of this post.
The point is that BizTalk CAN validate your message, but it does not do so by default. One of the reasons for this is flexibility: you may prefer to do the validation later in your process, perhaps in an orchestration, where you can handle the validation errors in your own way; or you may want the receive pipeline to do the validation and have BizTalk route the failed message. This decision is not made for you by BizTalk … your solution design will determine the appropriate implementation.
In this post I will present two ways in which you can get BizTalk to fully validate your message against its schema as early in the receiving process as possible: at the receive pipeline.
When using the standard XMLReceive pipeline in a receive location you can click on the ellipsis button to the right of the pipeline selection box to display the "Configure Pipeline" window. The screenshot below shows how the XMLReceive pipeline makes use of the XML disassembler pipeline component and the Party resolution pipeline component. When first setting up the receive location the ValidateDocument and DocumentSpecNames properties are False and blank respectively. In this default configuration the XMLReceive pipeline will try and match the received XML document’s target namespace and root node name to the published schemas in BizTalk, and the received document’s structure will be validated against the schema’s structure. No data validation is performed on the received document, however.
By setting the ValidateDocument property to True the XML disassembler pipeline component will be instructed to validate the data contained in the received document as well. Setting this property to true also requires that at least one DocumentSpecName is also provided, as the XML disassembler uses the schema specified by the fully qualified assembly name in the DocumentSpecNames property to identify the schema with which to perform the validation. With these properties set, if the receive location processes a message that fails validation an entry like the one below will be logged to the event log.
While this approach achieves the objective it falls short for an ideal solution, because:
This leads on to the second method to ensure that BizTalk validates the content of the received message: a custom pipeline that can be used in any receive location to do full schema validation, without requiring any additional configuration.
The concept behind the XMLValidatingReceive pipeline is to have a generic pipeline that one can use in any project, and in any receive location where the received document’s structure and data content needs to be validated against an existing schema. As the standard functionality of matching a received document with a deployed schema is still required, the XMLReceive pipeline is an excellent starting point. As the XMLReceive pipeline uses the XML disassembler and Party resolution pipeline components, the XMLValidatingReceive pipeline will need to use these two pipeline components as a starting point as well. In addition to these components, the XMLValidatingReceive pipeline will also make use if the XML validator pipeline component.
To create the XMLValidatingReceive pipeline, create a new receive pipeline. I created this pipeline in a generic BizTalk solution and project, so that I could use it for any solution, at any customer. In the new receive pipeline, add the XML disassembler, XML validator and Party resolution pipeline components, as per the screenshot below.
There is no need to change any of the default properties for any of these components. The pipeline can now be deployed and used in any BizTalk application. A message processed by this pipeline will now use the XML disassembler’s schema resolution functionality and it will validate the structure of the document against the resolved schema. The XML validator component will then use the resolved schema name to validate the received document’s data content, in accordance with any facets defined in the schema. Where a received document fails validation, the same error as in the previous method is shown (as below), except that the pipeline referred to will now be the new custom pipeline.
The result is a common pipeline that can be used in any receive location, and which does not require any configuration of pipeline properties to ensure that the received document is full validated. At the same time, this pipeline does not require any changes to be effected in the event of a change to a schema.
In conclusion, don’t assume that BizTalk will enforce those facets you diligently applied to your schema. You need to instruct BizTalk to validate a received message against the data constraints you define in your schema, and there are two simple ways in which this can be achieved. If it is a regular requirement to ensure that the received document is fully validated against the schema, then create a custom XMLValidatingReceive pipeline that can be used in any BizTalk development project.