by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
As most of you know, during the day I’m a mild-mannered .NET developer using all things Microsoft. But I lead a double life!
Before my (still undiagnosed) illness came upon me suddenly last year, most of my free time was spent with my family or playing golf at some of the country's best daily-fee courses here in Houston. I really enjoyed the fresh air, exercise and competitive nature of the sport. Unfortunately, my current condition makes playing golf almost impossible and much too risky. This situation has left me with much more free time than I'm normally used to having and after several months of rest and recuperation, I've decided to do some "after-hours" consulting.
Now in the evenings and on the weekends I do freelance development work using mostly “other technologies” such as Javascript, Ajax, PHP and mySQL on OS X Leopard. I also do some freelance photography in my local area and occasionally work on native and web iPhone applications. Since this part-time activity has begun to generate income, I've also started a new part-time company called Jeff Lynch Development, Ltd. and launched a new blog at http://blog.jefflynchdev.com (wordpress.com) to post about Mac & iPhone development, my new business and life in general.
I still plan to post regularly on this blog about all things Microsoft, so don't fret! But if you're like many of us and have a foot on both sides of the Microsoft and Apple fence, please join me at http://blog.jefflynchdev.com and have a good read! I'll also let you know when my new web site is up and running!
Currently reading: Inside Steve's Brain by Leander Kahney
by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
The ability to rigidly define the content of a data field is an essential element in any structured data definition. It is no surprise therefore one can define data constraints on elements and attributes in a schema through the means of various facets, such as enumerations, min length, regular expressions, and many more. The implementation of facets in a schema provides a large amount of value in terms of the data aspect of the contract to be used in an integration. Whether the integration is WCF-based or not, this data contract must be adhered to and messages that do not adhere to this contract must not be processed.
There is, however, no value in defining the data constraints if these constraints can not be enforced at the time the data is received, as allowing "invalid" data to be processed will invariably lead to errors in the processing of that data. It is for this reason that validating a message against its schema is oftentimes a necessary requirement. Some might say that this validation should be performed by default, and any messages that do not validate should be rejected as early in the processing of the message as possible. BizTalk does not do this by default.
At this point you may be saying what do you mean "BizTalk does not validate my message?" … hence, the title of this post.
The point is that BizTalk CAN validate your message, but it does not do so by default. One of the reasons for this is flexibility: you may prefer to do the validation later in your process, perhaps in an orchestration, where you can handle the validation errors in your own way; or you may want the receive pipeline to do the validation and have BizTalk route the failed message. This decision is not made for you by BizTalk … your solution design will determine the appropriate implementation.
In this post I will present two ways in which you can get BizTalk to fully validate your message against its schema as early in the receiving process as possible: at the receive pipeline.
Using the XMLReceive Pipeline
When using the standard XMLReceive pipeline in a receive location you can click on the ellipsis button to the right of the pipeline selection box to display the "Configure Pipeline" window. The screenshot below shows how the XMLReceive pipeline makes use of the XML disassembler pipeline component and the Party resolution pipeline component. When first setting up the receive location the ValidateDocument and DocumentSpecNames properties are False and blank respectively. In this default configuration the XMLReceive pipeline will try and match the received XML document’s target namespace and root node name to the published schemas in BizTalk, and the received document’s structure will be validated against the schema’s structure. No data validation is performed on the received document, however.
By setting the ValidateDocument property to True the XML disassembler pipeline component will be instructed to validate the data contained in the received document as well. Setting this property to true also requires that at least one DocumentSpecName is also provided, as the XML disassembler uses the schema specified by the fully qualified assembly name in the DocumentSpecNames property to identify the schema with which to perform the validation. With these properties set, if the receive location processes a message that fails validation an entry like the one below will be logged to the event log.
While this approach achieves the objective it falls short for an ideal solution, because:
- The XML disassembler pipeline component has logic built into it to identify the matching schema for a received message, so why does it not use this schema to perform validation? In other words, I would like to be able to change the Validate flag to true, without needing to specify a value for the DocumentSpecNames property.
- Every time the schema changes (possibly due to version changes) the developer will need to remember to go and change the configuration of the XMLReceive pipeline to ensure that the correct fully qualified assembly name is stored in the DocumentSpecNames property.
- If I have multiple receive locations that each need to do schema validation the previous task would be compounded as I would need to remember to set the properties for each receive location.
This leads on to the second method to ensure that BizTalk validates the content of the received message: a custom pipeline that can be used in any receive location to do full schema validation, without requiring any additional configuration.
Creating and Using the XMLValidatingReceive Pipeline
The concept behind the XMLValidatingReceive pipeline is to have a generic pipeline that one can use in any project, and in any receive location where the received document’s structure and data content needs to be validated against an existing schema. As the standard functionality of matching a received document with a deployed schema is still required, the XMLReceive pipeline is an excellent starting point. As the XMLReceive pipeline uses the XML disassembler and Party resolution pipeline components, the XMLValidatingReceive pipeline will need to use these two pipeline components as a starting point as well. In addition to these components, the XMLValidatingReceive pipeline will also make use if the XML validator pipeline component.
To create the XMLValidatingReceive pipeline, create a new receive pipeline. I created this pipeline in a generic BizTalk solution and project, so that I could use it for any solution, at any customer. In the new receive pipeline, add the XML disassembler, XML validator and Party resolution pipeline components, as per the screenshot below.
There is no need to change any of the default properties for any of these components. The pipeline can now be deployed and used in any BizTalk application. A message processed by this pipeline will now use the XML disassembler’s schema resolution functionality and it will validate the structure of the document against the resolved schema. The XML validator component will then use the resolved schema name to validate the received document’s data content, in accordance with any facets defined in the schema. Where a received document fails validation, the same error as in the previous method is shown (as below), except that the pipeline referred to will now be the new custom pipeline.
The result is a common pipeline that can be used in any receive location, and which does not require any configuration of pipeline properties to ensure that the received document is full validated. At the same time, this pipeline does not require any changes to be effected in the event of a change to a schema.
Summary
In conclusion, don’t assume that BizTalk will enforce those facets you diligently applied to your schema. You need to instruct BizTalk to validate a received message against the data constraints you define in your schema, and there are two simple ways in which this can be achieved. If it is a regular requirement to ensure that the received document is fully validated against the schema, then create a custom XMLValidatingReceive pipeline that can be used in any BizTalk development project.
by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
The ability to rigidly define the content of a data field is an essential element in any structured data definition. It is no surprise therefore one can define data constraints on elements and attributes in a schema through the means of various facets, such as enumerations, min length, regular expressions, and many more. The implementation of facets in a schema provides a large amount of value in terms of the data aspect of the contract to be used in an integration. Whether the integration is WCF-based or not, this data contract must be adhered to and messages that do not adhere to this contract must not be processed.
There is, however, no value in defining the data constraints if these constraints can not be enforced at the time the data is received, as allowing "invalid" data to be processed will invariably lead to errors in the processing of that data. It is for this reason that validating a message against its schema is oftentimes a necessary requirement. Some might say that this validation should be performed by default, and any messages that do not validate should be rejected as early in the processing of the message as possible. BizTalk does not do this by default.
At this point you may be saying what do you mean "BizTalk does not validate my message?" … hence, the title of this post.
The point is that BizTalk CAN validate your message, but it does not do so by default. One of the reasons for this is flexibility: you may prefer to do the validation later in your process, perhaps in an orchestration, where you can handle the validation errors in your own way; or you may want the receive pipeline to do the validation and have BizTalk route the failed message. This decision is not made for you by BizTalk … your solution design will determine the appropriate implementation.
In this post I will present two ways in which you can get BizTalk to fully validate your message against its schema as early in the receiving process as possible: at the receive pipeline.
Using the XMLReceive Pipeline
When using the standard XMLReceive pipeline in a receive location you can click on the ellipsis button to the right of the pipeline selection box to display the "Configure Pipeline" window. The screenshot below shows how the XMLReceive pipeline makes use of the XML disassembler pipeline component and the Party resolution pipeline component. When first setting up the receive location the ValidateDocument and DocumentSpecNames properties are False and blank respectively. In this default configuration the XMLReceive pipeline will try and match the received XML document’s target namespace and root node name to the published schemas in BizTalk, and the received document’s structure will be validated against the schema’s structure. No data validation is performed on the received document, however.
By setting the ValidateDocument property to True the XML disassembler pipeline component will be instructed to validate the data contained in the received document as well. Setting this property to true also requires that at least one DocumentSpecName is also provided, as the XML disassembler uses the schema specified by the fully qualified assembly name in the DocumentSpecNames property to identify the schema with which to perform the validation. With these properties set, if the receive location processes a message that fails validation an entry like the one below will be logged to the event log.
While this approach achieves the objective it falls short for an ideal solution, because:
- The XML disassembler pipeline component has logic built into it to identify the matching schema for a received message, so why does it not use this schema to perform validation? In other words, I would like to be able to change the Validate flag to true, without needing to specify a value for the DocumentSpecNames property.
- Every time the schema changes (possibly due to version changes) the developer will need to remember to go and change the configuration of the XMLReceive pipeline to ensure that the correct fully qualified assembly name is stored in the DocumentSpecNames property.
- If I have multiple receive locations that each need to do schema validation the previous task would be compounded as I would need to remember to set the properties for each receive location.
This leads on to the second method to ensure that BizTalk validates the content of the received message: a custom pipeline that can be used in any receive location to do full schema validation, without requiring any additional configuration.
Creating and Using the XMLValidatingReceive Pipeline
The concept behind the XMLValidatingReceive pipeline is to have a generic pipeline that one can use in any project, and in any receive location where the received document’s structure and data content needs to be validated against an existing schema. As the standard functionality of matching a received document with a deployed schema is still required, the XMLReceive pipeline is an excellent starting point. As the XMLReceive pipeline uses the XML disassembler and Party resolution pipeline components, the XMLValidatingReceive pipeline will need to use these two pipeline components as a starting point as well. In addition to these components, the XMLValidatingReceive pipeline will also make use if the XML validator pipeline component.
To create the XMLValidatingReceive pipeline, create a new receive pipeline. I created this pipeline in a generic BizTalk solution and project, so that I could use it for any solution, at any customer. In the new receive pipeline, add the XML disassembler, XML validator and Party resolution pipeline components, as per the screenshot below.
There is no need to change any of the default properties for any of these components. The pipeline can now be deployed and used in any BizTalk application. A message processed by this pipeline will now use the XML disassembler’s schema resolution functionality and it will validate the structure of the document against the resolved schema. The XML validator component will then use the resolved schema name to validate the received document’s data content, in accordance with any facets defined in the schema. Where a received document fails validation, the same error as in the previous method is shown (as below), except that the pipeline referred to will now be the new custom pipeline.
The result is a common pipeline that can be used in any receive location, and which does not require any configuration of pipeline properties to ensure that the received document is full validated. At the same time, this pipeline does not require any changes to be effected in the event of a change to a schema.
Summary
In conclusion, don’t assume that BizTalk will enforce those facets you diligently applied to your schema. You need to instruct BizTalk to validate a received message against the data constraints you define in your schema, and there are two simple ways in which this can be achieved. If it is a regular requirement to ensure that the received document is fully validated against the schema, then create a custom XMLValidatingReceive pipeline that can be used in any BizTalk development project.
by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
Michele recently wrote a lengthy but deliciously thorough article on MSDN (hat tip: Lynn) called Application Deployment Strategies which identifies five core scenarios for using Windows Communication Foundation within a distributed application. Those scenarios include:
Enterprise web services. Think secure, interoperable services that may also utilize new-ish WS* standards such as WS-AtomicTransaction and WS-ReliableMessaging.
Web 2.0 services. […]
by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
Michele recently wrote a lengthy but deliciously thorough article on MSDN (hat tip: Lynn) called Application Deployment Strategies which identifies five core scenarios for using Windows Communication Foundation within a distributed application. Those scenarios include:
Enterprise web services. Think secure, interoperable services that may also utilize new-ish WS* standards such as WS-AtomicTransaction and WS-ReliableMessaging.
Web 2.0 services. […]
by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
Another new WCF feature that’s part of .NET 3.5 SP1 has to do with better support for object references. DataContractSerializer has always supported serializing object references and dealing with graphs, including cycles, and not just simple trees. But doing so is not the default behavior — you have to tell DataContractSerializer that you want it to preserve object references when you instantiate it.
Let’s look at a simple example. Supposed that you have the following cyclic object graph (I’m assuming the same Person type that I used in my previous post):
Person p = new Person();
p.Id = “123”;
p.Name = “Aaron”;
p.Spouse = new Person();
p.Spouse.Id = “456”;
p.Spouse.Name = “Monica”;
p.Spouse.Spouse = p;
…
And now let’s supposed that you want to serialize it. If you create the DataContractSerializer using the default constructor, it will throw an exception when it identifies the cycle during serialization. However, you can tell DataContractSerializer to preserve object references using one of the other constructors:
DataContractSerializer dcs = new DataContractSerializer(typeof(Person),
null, int.MaxValue, false, true /* preserve object refs */, null);
using (FileStream fs = new FileStream(“person.xml”, FileMode.Create))
{
dcs.WriteObject(fs, p);
}
The resulting person.xml file now looks like this:
<Person z:Id=“1“ xmlns=“http://schemas.datacontract.org/2004/07/SerializationSp1“ xmlns:i=“http://www.w3.org/2001/XMLSchema-instance“ xmlns:z=“http://schemas.microsoft.com/2003/10/Serialization/“>
<Id z:Id=“2“>123</Id>
<Name z:Id=“3“>Aaron</Name>
<Spouse z:Id=“4“>
<Id z:Id=“5“>456</Id>
<Name z:Id=“6“>Monica</Name>
<Spouse z:Ref=“1“ i:nil=“true“/>
</Spouse>
</Person>
Notice that each reference type has been given an “Id” attribute and the nested Spouse reference refers back to the containing Person via the Ref attribute, thereby preserving the references within the XML.
Now, as of SP1, the definitions for the Id/Ref attributes are now part of the generated schema. If you run SvcUtil.exe /dconly over the assembly containing Person, it will produce a schema file for the “http://schemas.microsoft.com/2003/10/Serialization” namespace. And within that schema, you’ll find the following definitions for Id/Ref, which are defined as ID/IDREF types:
…
<xs:attribute name=“Id“ type=“xs:ID“ />
<xs:attribute name=“Ref“ type=“xs:IDREF“ />
…
ID and IDREF are standard DTD/XSD types that are widely supported across platforms.
One problem with employing this object-reference-preservation technique is that you don’t have direct control over how the DataContractSerializer is constructed when defining your WCF services. You can, however, implement a behavior that intercepts the standard serializer creation process so that you can enable this feature. Sowmy provides a complete example of how to accomplish this over on his blog.
by community-syndication | May 14, 2008 | BizTalk Community Blogs via Syndication
Another new WCF feature that’s part of .NET 3.5 SP1 has to do with better support for object references. DataContractSerializer has always supported serializing object references and dealing with graphs, including cycles, and not just simple trees. But doing so is not the default behavior — you have to tell DataContractSerializer that you want it to preserve object references when you instantiate it.
Let’s look at a simple example. Supposed that you have the following cyclic object graph (I’m assuming the same Person type that I used in my previous post):
Person p = new Person();
p.Id = “123”;
p.Name = “Aaron”;
p.Spouse = new Person();
p.Spouse.Id = “456”;
p.Spouse.Name = “Monica”;
p.Spouse.Spouse = p;
…
And now let’s supposed that you want to serialize it. If you create the DataContractSerializer using the default constructor, it will throw an exception when it identifies the cycle during serialization. However, you can tell DataContractSerializer to preserve object references using one of the other constructors:
DataContractSerializer dcs = new DataContractSerializer(typeof(Person),
null, int.MaxValue, false, true /* preserve object refs */, null);
using (FileStream fs = new FileStream(“person.xml”, FileMode.Create))
{
dcs.WriteObject(fs, p);
}
The resulting person.xml file now looks like this:
<Person z:Id=“1“ xmlns=“http://schemas.datacontract.org/2004/07/SerializationSp1“ xmlns:i=“http://www.w3.org/2001/XMLSchema-instance“ xmlns:z=“http://schemas.microsoft.com/2003/10/Serialization/“>
<Id z:Id=“2“>123</Id>
<Name z:Id=“3“>Aaron</Name>
<Spouse z:Id=“4“>
<Id z:Id=“5“>456</Id>
<Name z:Id=“6“>Monica</Name>
<Spouse z:Ref=“1“ i:nil=“true“/>
</Spouse>
</Person>
Notice that each reference type has been given an “Id” attribute and the nested Spouse reference refers back to the containing Person via the Ref attribute, thereby preserving the references within the XML.
Now, as of SP1, the definitions for the Id/Ref attributes are now part of the generated schema. If you run SvcUtil.exe /dconly over the assembly containing Person, it will produce a schema file for the “http://schemas.microsoft.com/2003/10/Serialization” namespace. And within that schema, you’ll find the following definitions for Id/Ref, which are defined as ID/IDREF types:
…
<xs:attribute name=“Id“ type=“xs:ID“ />
<xs:attribute name=“Ref“ type=“xs:IDREF“ />
…
ID and IDREF are standard DTD/XSD types that are widely supported across platforms.
One problem with employing this object-reference-preservation technique is that you don’t have direct control over how the DataContractSerializer is constructed when defining your WCF services. You can, however, implement a behavior that intercepts the standard serializer creation process so that you can enable this feature. Sowmy provides a complete example of how to accomplish this over on his blog.
by community-syndication | May 13, 2008 | BizTalk Community Blogs via Syndication
I came across an unusual requirement earlier this year while at a leading golfing industry company: We needed to retrieve current tournament standings from the PGA Web site, pulling scores and statistics from them by FTP every 2 minutes while the tournaments were in progress. Ultimately, the standings would trickle through to a data-driven Web site. The interesting part here was the pickup location, there was a predefined FTP folder structure, with a separate folder for each tournament (the files were always the same name, but the locations would vary). We could have potentially multiple receive locations that become active during a given week, but only for a finite period of time, yet all pointing at different FTP locations. BizTalk has dynamic send ports of course, but no dynamic receive locations. How to handle this? Clearly, although manually managing all this was possible, it really wasn’t a viable solution given the amount of on-going effort it would require.
I found out that there was an overall tournament schedule file that was posted on a weekly basis, and that’s how I solved it. The solution basically:
- retrieves a schedule file (on a weekly basis) and publishes the schedule to the message box
- a CreateReveiveLocations orchestration subscribes to schedule files, and picks it up
- the orchestration maps the schedule to a structure that is easier to work with (we map inside the orchestration as we thought we may have other interesting things we could do with the schedule file in a future release)
- the orchestration removes any previously-created dynamic receive locations (based on a standard naming pattern, and they all belong to a known receive port)
- the orchestration calls a helper class that creates all the FTP receive locations. We know when the tournament starts, so:
- we only create receive locations for current and future tournaments
- we set the start date for the tournament to be the day of the tournament, so no polling will happen before the tournament starts
How cool is that? It’s very cool to drop an XML document into BizTalk, and watch it create a bunch (and there are a LOT) of receive locations that it will subsequently use to drive a process. I’m all in favor of self-configuring environments!
We were in POC mode, and this took me a couple of days to get it all done (using good naming conventions, etc of course), and get it all working end-to-end. I’m sure I could have come up with a pretty good way to do this just in code, but it would have taken a LOT longer that 2 days.
Here’s what the orchestration looks like:
There are two calls to a helper class above. The first removes any receive locations that were previously created (I use a naming convention and a unique name suffix, so I know which ones are dynamic). The second call passes in all the information required to create the new batch of receive locations.
The schedule file contained a tournament date. Because of that, we can skip creating FTP receives for tournaments that have ended.
Drilling into the CreateFtpReceiveLocation code, this snippet :
- first gets a reference to the receive port (which has a well know name)
- creates a new receive location
- sets the transport type to FTP
Note that if this were not a POC, I would have cached the ReceiveHandler and done other optimizations, however, even without optimizations, this creates hundreds of receive locations in seconds.
This last snippet:
- assigns the pipeline to use for this receive location
- set receive locations
- saves the changes to the BizTalk management database
There you have it. With a trivial amount of code, I eliminated what would have been significant ongoing manual intervention in the process. I fully automated what would have been a very tedious, and error-prone, task. The client was thrilled as this was a very elegant solution to an ongoing problem.
For anyone after the code rather that pretty pictures of it :), please see below.
Technorati Tags: BizTalk,SOA,Dynamic Receive,ExplorerOM
—————————–
public static class LocationManagement
{
const string CONNECTION_STRING = “Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=BizTalkMgmtDb;Server=(local)”;
public static void CreateFtpReceiveLocations(string receivePortName, XmlDocument receiveLocations, string prefix, string server, string addressTemplate, string uid, string pwd)
{
XmlNodeList locs = receiveLocations.SelectNodes(“//ReceiveLocation”);
DateTime cutoffDate = System.DateTime.Now.Add(new TimeSpan(-7,0,0,0));
foreach (XmlNode loc in locs)
{
// if start date is more than 7 days ago, we don’t need this one any more
if (DateTime.Parse(loc.Attributes[“StartDate”].Value) > cutoffDate)
{
CreateFtpReceiveLocation(receivePortName, prefix + loc.Attributes[“Address”].Value, string.Format(addressTemplate, loc.Attributes[“Address”].Value), server, uid, pwd, DateTime.Parse(loc.Attributes[“StartDate”].Value));
}
}
}
public static bool CreateFtpReceiveLocation(string receivePortName, string receiveLocationName, string address, string server, string uid, string pwd, DateTime startDate)
{
try
{
BtsCatalogExplorer root = new BtsCatalogExplorer();
root.ConnectionString = CONNECTION_STRING;
//GetReceive Port
ReceivePort receivePort = root.ReceivePorts[receivePortName];
//Create a new receive location and add it to the receive port
ReceiveLocation myreceiveLocation = receivePort.AddNewReceiveLocation(); ;
myreceiveLocation.Address = address;
myreceiveLocation.Name = receiveLocationName;
//Receive Handler
foreach (ReceiveHandler handler in root.ReceiveHandlers)
{
if (handler.TransportType.Name == “FTP”)
{
myreceiveLocation.ReceiveHandler = handler;
break;
}
}
//Associate a transport protocol and URI with the receive location.
ProtocolType protocol = root.ProtocolTypes[“FTP”];
myreceiveLocation.TransportType = protocol;
Pipeline pipeline = root.Pipelines[“Microsoft.BizTalk.DefaultPipelines.PassThruReceive”];
myreceiveLocation.ReceivePipeline = pipeline;
myreceiveLocation.StartDate = startDate;
myreceiveLocation.StartDateEnabled = true;
string ReceiveLocationTransportTypeData = “<CustomProps><AdapterConfig vt=\”8\”><Config xmlns:xsi=\”http://www.w3.org/2001/XMLSchema-instance\” xmlns:xsd=\”http://www.w3.org/2001/XMLSchema\”><uri>ftp://BTS2006Dev:21/ClientName.PGA/*.xml</uri><serverAddress>BTS2006Dev</serverAddress><serverPort>21</serverPort><userName>administrator</userName><password>******</password><accountName /><fileMask>*.xml</fileMask><targetFolder>ClientName.PGA</targetFolder><representationType>binary</representationType><maximumBatchSize>0</maximumBatchSize><maximumNumberOfFiles>0</maximumNumberOfFiles><passiveMode>False</passiveMode><firewallType>NoFirewall</firewallType><firewallPort>21</firewallPort><pollingUnitOfMeasure>Seconds</pollingUnitOfMeasure><pollingInterval>60</pollingInterval><errorThreshold>10</errorThreshold><maxFileSize>100</maxFileSize></Config></AdapterConfig></CustomProps>”;
XmlDocument docTransportTypeData = new XmlDocument();
docTransportTypeData.LoadXml(System.Web.HttpUtility.HtmlDecode(ReceiveLocationTransportTypeData));
docTransportTypeData.SelectSingleNode(“//userName”).InnerText = uid;
docTransportTypeData.SelectSingleNode(“//password”).InnerText = pwd;
docTransportTypeData.SelectSingleNode(“//uri”).InnerText = address;
docTransportTypeData.SelectSingleNode(“//serverAddress”).InnerText = server;
myreceiveLocation.TransportTypeData = ReceiveLocationTransportTypeData;
//Enable the receive location.
myreceiveLocation.Enable = true;
//Save changes
root.SaveChanges();
return true;
}
catch (Exception ex)
{
throw new Exception(“Error creating receive location”, ex);
}
}
by community-syndication | May 13, 2008 | BizTalk Community Blogs via Syndication
I came across an unusual requirement earlier this year while at a leading golfing industry company: We needed to retrieve current tournament standings from the PGA Web site, pulling scores and statistics from them by FTP every 2 minutes while the tournaments were in progress. Ultimately, the standings would trickle through to a data-driven Web site. The interesting part here was the pickup location, there was a predefined FTP folder structure, with a separate folder for each tournament (the files were always the same name, but the locations would vary). We could have potentially multiple receive locations that become active during a given week, but only for a finite period of time, yet all pointing at different FTP locations. BizTalk has dynamic send ports of course, but no dynamic receive locations. How to handle this? Clearly, although manually managing all this was possible, it really wasn’t a viable solution given the amount of on-going effort it would require.
I found out that there was an overall tournament schedule file that was posted on a weekly basis, and that’s how I solved it. The solution basically:
- retrieves a schedule file (on a weekly basis) and publishes the schedule to the message box
- a CreateReveiveLocations orchestration subscribes to schedule files, and picks it up
- the orchestration maps the schedule to a structure that is easier to work with (we map inside the orchestration as we thought we may have other interesting things we could do with the schedule file in a future release)
- the orchestration removes any previously-created dynamic receive locations (based on a standard naming pattern, and they all belong to a known receive port)
- the orchestration calls a helper class that creates all the FTP receive locations. We know when the tournament starts, so:
- we only create receive locations for current and future tournaments
- we set the start date for the tournament to be the day of the tournament, so no polling will happen before the tournament starts
How cool is that? It’s very cool to drop an XML document into BizTalk, and watch it create a bunch (and there are a LOT) of receive locations that it will subsequently use to drive a process. I’m all in favor of self-configuring environments!
We were in POC mode, and this took me a couple of days to get it all done (using good naming conventions, etc of course), and get it all working end-to-end. I’m sure I could have come up with a pretty good way to do this just in code, but it would have taken a LOT longer that 2 days.
Here’s what the orchestration looks like:
There are two calls to a helper class above. The first removes any receive locations that were previously created (I use a naming convention and a unique name suffix, so I know which ones are dynamic). The second call passes in all the information required to create the new batch of receive locations.
The schedule file contained a tournament date. Because of that, we can skip creating FTP receives for tournaments that have ended.
Drilling into the CreateFtpReceiveLocation code, this snippet :
- first gets a reference to the receive port (which has a well know name)
- creates a new receive location
- sets the transport type to FTP
Note that if this were not a POC, I would have cached the ReceiveHandler and done other optimizations, however, even without optimizations, this creates hundreds of receive locations in seconds.
This last snippet:
- assigns the pipeline to use for this receive location
- set receive locations
- saves the changes to the BizTalk management database
There you have it. With a trivial amount of code, I eliminated what would have been significant ongoing manual intervention in the process. I fully automated what would have been a very tedious, and error-prone, task. The client was thrilled as this was a very elegant solution to an ongoing problem.
For anyone after the code rather that pretty pictures of it :), please see below.
Technorati Tags: BizTalk,SOA,Dynamic Receive,ExplorerOM
—————————–
public static class LocationManagement
{
const string CONNECTION_STRING = “Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=BizTalkMgmtDb;Server=(local)”;
public static void CreateFtpReceiveLocations(string receivePortName, XmlDocument receiveLocations, string prefix, string server, string addressTemplate, string uid, string pwd)
{
XmlNodeList locs = receiveLocations.SelectNodes(“//ReceiveLocation”);
DateTime cutoffDate = System.DateTime.Now.Add(new TimeSpan(-7,0,0,0));
foreach (XmlNode loc in locs)
{
// if start date is more than 7 days ago, we don’t need this one any more
if (DateTime.Parse(loc.Attributes[“StartDate”].Value) > cutoffDate)
{
CreateFtpReceiveLocation(receivePortName, prefix + loc.Attributes[“Address”].Value, string.Format(addressTemplate, loc.Attributes[“Address”].Value), server, uid, pwd, DateTime.Parse(loc.Attributes[“StartDate”].Value));
}
}
}
public static bool CreateFtpReceiveLocation(string receivePortName, string receiveLocationName, string address, string server, string uid, string pwd, DateTime startDate)
{
try
{
BtsCatalogExplorer root = new BtsCatalogExplorer();
root.ConnectionString = CONNECTION_STRING;
//GetReceive Port
ReceivePort receivePort = root.ReceivePorts[receivePortName];
//Create a new receive location and add it to the receive port
ReceiveLocation myreceiveLocation = receivePort.AddNewReceiveLocation(); ;
myreceiveLocation.Address = address;
myreceiveLocation.Name = receiveLocationName;
//Receive Handler
foreach (ReceiveHandler handler in root.ReceiveHandlers)
{
if (handler.TransportType.Name == “FTP”)
{
myreceiveLocation.ReceiveHandler = handler;
break;
}
}
//Associate a transport protocol and URI with the receive location.
ProtocolType protocol = root.ProtocolTypes[“FTP”];
myreceiveLocation.TransportType = protocol;
Pipeline pipeline = root.Pipelines[“Microsoft.BizTalk.DefaultPipelines.PassThruReceive”];
myreceiveLocation.ReceivePipeline = pipeline;
myreceiveLocation.StartDate = startDate;
myreceiveLocation.StartDateEnabled = true;
string ReceiveLocationTransportTypeData = “<CustomProps><AdapterConfig vt=\”8\”><Config xmlns:xsi=\”http://www.w3.org/2001/XMLSchema-instance\” xmlns:xsd=\”http://www.w3.org/2001/XMLSchema\”><uri>ftp://BTS2006Dev:21/ClientName.PGA/*.xml</uri><serverAddress>BTS2006Dev</serverAddress><serverPort>21</serverPort><userName>administrator</userName><password>******</password><accountName /><fileMask>*.xml</fileMask><targetFolder>ClientName.PGA</targetFolder><representationType>binary</representationType><maximumBatchSize>0</maximumBatchSize><maximumNumberOfFiles>0</maximumNumberOfFiles><passiveMode>False</passiveMode><firewallType>NoFirewall</firewallType><firewallPort>21</firewallPort><pollingUnitOfMeasure>Seconds</pollingUnitOfMeasure><pollingInterval>60</pollingInterval><errorThreshold>10</errorThreshold><maxFileSize>100</maxFileSize></Config></AdapterConfig></CustomProps>”;
XmlDocument docTransportTypeData = new XmlDocument();
docTransportTypeData.LoadXml(System.Web.HttpUtility.HtmlDecode(ReceiveLocationTransportTypeData));
docTransportTypeData.SelectSingleNode(“//userName”).InnerText = uid;
docTransportTypeData.SelectSingleNode(“//password”).InnerText = pwd;
docTransportTypeData.SelectSingleNode(“//uri”).InnerText = address;
docTransportTypeData.SelectSingleNode(“//serverAddress”).InnerText = server;
myreceiveLocation.TransportTypeData = ReceiveLocationTransportTypeData;
//Enable the receive location.
myreceiveLocation.Enable = true;
//Save changes
root.SaveChanges();
return true;
}
catch (Exception ex)
{
throw new Exception(“Error creating receive location”, ex);
}
}
by community-syndication | May 13, 2008 | BizTalk Community Blogs via Syndication
WCF: Technique of debugging inconsistency in Wsdl and Response messages.
I have consumed severalWCF services whereWSDLs do not conform the Response messagess (See below. I’ve bolded the sample text related to one issue.).We canfix the issues by changing proxy code. It’s not a big issue but now we must manually change proxy every time we have updated the proxy of this service.
Here the checklist howwe debug this case:
[I’ve created the proxy for the web-service using any method in Visual Studio 2008 like “Add Service Reference” command.]
1. We have found that the client code cannot get all data from the service. Some data are lost.
2. Using SoapUI make sure the Response returned all data. This is a sign of this inconsistency!
3. Using debugger we havefound that data are lost exactly in
response = client.Method(request)
line in the client code.
What does it mean? The data is successfully returned in the Response message. But the client proxy code cannot process it. When the proxy code is deserializing the response message some data are lost. SoapUI does not process returned data, it just shows them as raw text. My proxy code is trying to deserialize this text, convert text to objects in memory. [The deserializer is the code in the Microsoft libraries in the System.Runtime.Serialization namespace. There are two of them in WCF: XmlSerializer and DataContractSerializer.] Some of returned data disappeared during the Deserializing process. These cases are described in article “WCF: values disappeared in response: derived classes and serialization/deserialization order error” [http://geekswithblogs.net/LeonidGaneline/archive/2008/05/01/wcf-values-disappeared-in-response-derived-classes-and-serializationdeseriazlization-order.aspx]
4. We have to openthe proxy source code (maybe in Reference.cs file) and fix code (see below). That makes it inconsistent with Wsdl file and next time when you update this Web or Service reference you have to fix code again. But your client is fixed now!
5. Document this fix! And do not make it in proxy code. J
You are lucky if you have gotinconsistency in Response messages. The worst situation if inconsistency is in Request messages. There are only two options: the service does not care about these inconsistencies and you are lucky again, or service rejects your request with a fault message. Moreover I make sure this message does not give you any clue what is going on. Just try to get the “correct” request from any source.
[WSDL ]
<xsd:complexType name=“ArrayOf_xsd_string“>
<xsd:sequence>
<xsd:element minOccurs=“0“ maxOccurs=“unbounded“ name=“item“ type=“xsd:string“ />
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name=“PromoDetails“>
<xsd:sequence>
<xsd:element name=“assetTypes“ nillable=“true“ type=“impl:ArrayOf_xsd_string“ />
</xsd:sequence>
[Response message]
<assetTypes>
<assetTypes>Album</ assetTypes >
</assetTypes>
[Response message How it should be with regard to WSDL]
<assetTypes>
<item>Album</ item >
</assetTypes>
[Auto generated Proxy code]
private string[] assetTypesField;
[System.Xml.Serialization.XmlArrayAttribute(IsNullable=true, Order=1)]
[System.Xml.Serialization.XmlArrayItemAttribute(“item”, IsNullable = false)]
public string[] assetTypes {
get {
return this.assetTypesField;
}
set {
this.assetTypesField = value;
this.RaisePropertyChanged(“assetTypes”);
}
}
[Fixed Proxy code]
private string[] assetTypesField;
[System.Xml.Serialization.XmlArrayAttribute(IsNullable=true, Order=1)]
[System.Xml.Serialization.XmlArrayItemAttribute(“assetTypes “, IsNullable = false)]
public string[] assetTypes {
get {
return this.assetTypesField;
}
set {
this.assetTypesField = value;
this.RaisePropertyChanged(“assetTypes”);
}
}
