Enterprise Integration Pattern – Design Patterns Notes – Message Construction – Part 1

by vijaymstech | Aug 7, 2016 | BizTalk Community Blogs via Syndication

Enterprise Integration Pattern – Design Patterns Notes – Message Construction – Part 1

Enterprise Integration Pattern – Design Patterns Notes – Message Construction

Message Construction:

Command Message:[Command Pattern]

http://www.enterpriseintegrationpatterns.com/patterns/messaging/CommandMessage.html

How can messaging be used to invoke a procedure in another application?

Use a Command Message to reliably invoke a procedure in another application.

There is no specific message type for commands; a Command Message is simply a regular message that happens to contain a
command.

This will be used in SOA Patterns using WCF/Web Service or A2A scenario. You can achieve this by creating a Data Contract/Message Contract in WCF and specifying message details as well as message Operation also. Message operation meaning
how we are going to use the message body and what operation are we going to perform, whether Insert/Update/Delete probably to the Backend Systems. Mostly this kind of design patterns are similar to DML statements of TSQL Database.

Most of this are defined inside SOAP body while you communicate to the Server to do particular operation on the message.

Disadvantages are that it is kind of defined message structure, so any changes has to be done both at server as well as with client also.

Example:
InsertNewCustomer
DeleteOldCustomer
UpdateExistingCustomer

[Refer : SOA Patterns in BizTalk 2009.pdf – Page 114]

Document Message:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/DocumentMessage.html

How can messaging be used to transfer data between applications?

Use a Document Message to reliably transfer a data structure between applications.  Whereas a Command Message tells the receiver to invoke certain behavior, a Document Message just passes data and lets the receiver decide what, if anything, to do with the data. The data is a single unit of data, a single object or data structure
which may decompose into smaller units.

This is similar to Command Message, but it will not let the server/application knows, what kind of operation will be done on the message body. It will just transfer the data between applications. It will provide sutle information to the called applications, but how it interpret data is upto them. NewCustomer details in Application A is inserting new Customer to their system. But if it is passed on to another Application B, NewCustomer details may be to send a welcome kit for NewCustomer etc. How the message get interpreted is upto the Business need of the application that is built on.

Example:
PurchaseOrder created by Purchasing department application is to fulfill the order and this information gets passed on to the Account Department application is to start creating Invoice for received Purchase Order and send the bill to the client.

If you think beyond SOA architecture, this can be implemented in normal B2B scenario using different format like EDI/AS2, Rosattanet, SAP-IDOCs.

[Refer : SOA Patterns in BizTalk 2009.pdf – Page 115]

Event Message:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/EventMessage.html
How can messaging be used to transmit events from one application to another?

Use an Event Message for reliable, asynchronous event notification between applications.

When a subject has an event to announce, it will create an event object, wrap it in a message, and send it on a channel. The observer will receive the event message, get the event, and process it. Messaging does not change the event notification, just makes sure that the notification gets to the observer.

Event message based Message Construction design patterns are used to let other applications know that something happens and that will kick start series of other processes.

Example:

In B2B/SOA scenario, Purchase order gets created event will trigger Fullfillment Notification to Order FullFillment Department, inturn it will create Shipment Notification to Logistics department and so on.

[Refer : SOA Patterns in BizTalk 2009.pdf – Page 117]

Request – Reply:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/RequestReply.html

When two applications communicate via Messaging, the communication is one-way. The applications may want a two-way conversation.

https://msdn.microsoft.com/en-us/library/aa559029.aspx

When an application sends a message, how can it get a response from the receiver?

In a request/response messaging pattern, one party sends a request message and the receiving party returns a response message. Two typical examples of request/response processing are the interaction that a browser has with a Web server using the HTTP adapter, and Web service processing using the Simple Object Access Protocol (SOAP) adapter. In BizTalk Server, both the request and the response messages are handled in a typical publish/subscribe fashion. This is an important consideration to understand when you performance-tune a BizTalk application, because a system requiring high throughput might be configured differently than one requiring low latency for individual messages.

When a message is received by a request/response style receive adapter, BizTalk Server first publishes the request message to the MessageBox database. Next this message is received by the appropriate subscriber, which is likely an orchestration bound to a receive port. This subscriber formulates a response message and publishes it to the MessageBox, along with properties that cause it to be sent back to the receive port from which the request came. Finally, the response message is picked up by the publisher of the request, the receive adapter that submitted the request, and is returned to the calling application. The diagram below provides a detailed graphical representation of these steps.

Flow of request/response message received by SOAP adapter:

• The SOAP adapter submits messages to the Endpoint Manager.
• The Endpoint Manager publishes the message into the MessageBox.
• The orchestration, which is bound to the receive port and therefore has a subscription for the message, receives the message and processes it.
• The orchestration sends a response message that is published to the MessageBox.
• The Endpoint Manager receives the response message.
• The Endpoint Manager returns the response to the SOAP adapter

Return Address:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/ReturnAddress.html

The request message should contain a Return Address that indicates where to send the reply message.

The request message should contain a Return Address that indicates where to send the reply message.

Steps to create Return Address Design pattern in BizTalk:

Create a schema with structure of you business details like purchase order details and add return address http URL in it.

Create Orchestration which will be exposed as http WCF Service.

In Orchestration, start another orchestration and pass on the request message and finish the orchestration immediately.

In Child/started orchestration, process your message like inserting PO to the database, but if the PO Quantity cannot be fulfilled , send the PO change/Reject reply to the caller by dynamic send port using Return Address URL. For testing purpose, you can also use File port to send the message, or else you need to create a WCF Service and host it in Console/IIS to listen to your response message. If you are sending it to HTTP Dynamic Send port, you have to wait for the response message in the child orchestration.

Briefly considered Self-Correlating Port is similar to Return Address, but dismissed it. Self-Correlating Port is similar to asynchronous callback in WCF, but it is happening inside Single BizTalk Orchestration process. Both caller and callee are in residing in same process.

Even though it make sense to implement this kind of design patterns in SOA or web based scenario. If we eliminate the web based scenario, we already implementing this pattern in B2B scenarios. On second thought, may be not. Because in B2B scenarios, we will know static client response location and return adddress is not part of the message. This will work only return address should be part of the incoming message.

Thinking further, sending email response message to the consumer about their application status, may be considered as Return Address Design Patterns. Since consumer email is part of incoming message details.

Correlation Identifier:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/CorrelationIdentifier.html

Each reply message should contain a Correlation Identifier, a unique identifier that indicates which request message this reply is for.

https://msdn.microsoft.com/en-us/library/aa578692?f=255&MSPPError=-2147217396 https://blogs.msdn.microsoft.com/richardbpi/2006/05/01/biztalk-correlation-of-untyped-messages/
http://www.cloudcasts.net/ViewWebcast.aspx?webcastid=2521599734904944607

This is implemented in BizTalk using Correlation. Take for example, we will be receiving a Purchase Order from the client and based on the purchase order, we have to send back a Invoice. This can be achieved by the Promoting PO Number, and creating Correlation Set and Type using PO Number and PO message will be send Invoice Orchestration. Invoice Orchestration will create Invoice and send it back to PO orchestration based on the correlated PO Number.

Message Sequence:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageSequence.html

Whenever a large set of data may need to be broken into message-size chunks, send the data as a Message Sequence and mark each message with sequence identification fields.

https://blogs.msdn.microsoft.com/richardbpi/2006/05/08/biztalk-aggregation-pattern-for-large-batches/

Links on How to deal with Large Message Size in Biztalk:

https://msdn.microsoft.com/en-us/library/aa560481.aspx
https://blogs.msdn.microsoft.com/biztalkcpr/2008/11/04/tuning-large-message-threshold-and-fragment-size/
https://msdn.microsoft.com/en-us/library/aa547883.aspx
https://social.msdn.microsoft.com/Forums/en-US/f0eae8b8-b241-4b38-8726-1a4c4fdf51ed/handling-large-messages-in-biztalk-2010? forum=biztalkgeneral
http://social.technet.microsoft.com/wiki/contents/articles/7801.biztalk-server-performance-tuning-optimization.aspx
http://www.codeproject.com/Articles/180216/Transfer-Large-Files-using-BizTalk-Receive-Side
http://www.codeproject.com/Articles/180333/Transfer-Large-Files-using-BizTalk-Send-Side
http://www.codeproject.com/Articles/142171/Transfer-Extremely-Large-Files-Using-Windows-Servi
https://blogs.msdn.microsoft.com/biztalk_core_engine/2005/02/28/large-messages-in-biztalk-2004-whats-the-deal/

Message Expiration:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageExpiration.html

How can a sender indicate when a message should be considered stale and thus shouldn’t be processed?

Messaging practically guarantees that the Message will eventually be delivered to the receiver.  What it cannot guarantee is how long the delivery may take. For example, if the network connecting the sender and receiver is down for a week, then it could take a week to deliver a message. Messaging is highly reliable, even when the participants (sender, network, and receiver) are not, but messages can take a very long time to transmit in unreliable circumstances. (For more details, see Guaranteed Delivery.)

Example from Enterprise Intergration Patterns Book:

Often, a message’s contents have a practical limit for how long they’re useful. A caller issuing a stock quote request probably looses interest if it does not receive an answer within a minute or so.  That means the request should not take more than a minute to transmit, but also that the answer had better transmit back very quickly. A stock quote reply more than a minute or two old is probably too old and therefore unreliable.

Once the sender sends a message and does not get a reply, it has no way to cancel or recall the message. Likewise, a receiver could check when a message was sent and reject the message if it’s too old, but different senders under different circumstances may have different ideas about how long is too long, so how does the receiver know which messages to reject?

Example/Scenarios:

In BizTalk, you can handle this in 2 different ways I can think off. Both of them use correlation to implement this scenario.

Method 1:

Assuming, when we send message to BizTalk Orchestration, we also include message expiration as an element in the form of notice period. You can use XPath or distinguished field to get this value from xml element. Use correlation to send message to other application/component/Third Party, then we can use this expiration element in the message as waiting period in the listen shape to receive the request back, else close the instance and log an appropriate error message.

Method 2:

If you want to set the expiration date of a business process but not have an option in the xml or flat file message, there are few non-promoted properties available in message context property. FileCreationTime and AdapterReceiveComplete Time are
those 2 properties. To use them, these properties need to be promoted in the custom pipeline component. Both of them are shown using GMT time format and you need to add/subtract the difference in the hosted Biztalk server time zone and need to be
modified. Inside Orchestration use this properties to come up with the time for listen shape to receive the request back, else close the instance and log an appropriate error message.

Format Indicator:

http://www.enterpriseintegrationpatterns.com/patterns/messaging/FormatIndicator.html

How can a message’s data format be designed to allow for possible future changes?

Even when you design a data format that works for all participating applications, future requirements may change. New applications may be added that have new format requirements, new data may need to be added to the messages, or developers may find better ways to structure the same data. Whatever the case, designing a single enterprise data model is difficult enough; designing one that will never need to change in the future is darn near impossible.

When an enterprise’s data format changes, there would be no problem if all of the applications change with it. If every application stopped using the old format and started using the new format, and all did so at exactly the same time, then conversion would be simple. The problem is that some applications will be converted before others, while some less-used applications may never be converted at all. Even if all applications could be converted at the same time, all messages would have to be consumed so that all channels are empty before the conversion could occur.

Realistically, applications are going to have to be able to support the old format and the new format simultaneously. To do this, applications will need to be able to tell which messages follow the old format and which use the new.

https://www.packtpub.com/networking-and-servers/soa-patterns-biztalk-server-2009

Richard Seroter books reference given in this article mostly talks about versioning of the schemas and other artifacts, and it talks in detail about the Major and Minor Versioning changes. Those are all valid point but not able to understand how it will be related to data format.

Say for example I have 2 clients/customers, I need to send PO 850 with different EDI version namely 4010 and 4050. For those I need to execute two different maps, schemas, etc. Those I will term as data format, not the one with simple adding of xml
element in the schema.

If you add couple of extra fields in the message but it still need to be passed to orchestration and to send port, you can still do that by making ValidateDocument = false in the XML Receive Pipeline Property. By default it is false.

Check below article which talk about simple BizTalk Hack in detail.

https://masteringbiztalkserver.wordpress.com/2011/03/02/how-to-validate-incoming-messages-for-xml-structure-and-data-type-using-existing-xmlreceive-pipeline/
https://blogs.msdn.microsoft.com/nabeelp/2008/05/14/biztalk-does-not-validate-my-message/

BizTalk Self Correlating port

by vijaymstech | Aug 4, 2016 | BizTalk Community Blogs via Syndication

BizTalk Self Correlating port

Self Correlating is one of the type of Direct Binding in orchestration. This is mainly used for starting child orchestration from parent orchestration and get a response from the child orchestration. This will work similar to call orchestration, but more towards asynchronize ways and we will not be using any correlation to implement this. This works like a loop back adapter. To implement self correlating direct binding, you have to perform few steps.

Parent Orchestration:

In parent orchestration, create a self correlating receive port with message type which you will be receiving from child orchestration. Connect it to a receive shape to receive message.

Before receive shape, place an start orchestration shape and it will pass on few orchestration parameters to the child orchestration.

Child Orchestration:

In child orchestration, have few orchestration parameters as you required. Add a configured port parameter with send communication direction and it will be connected with Send shape in the child orchestration.

This child orchestration configured port parameter will have value of Parent orchestration self correlating port in the Start Orchestration shape.

Once you deployed the orchestration by your preferred method, binding will be little bit tricky. You will not see any Orchestration port binding for this self correlating ports both in Parent as well as child orchestration. It will seems like a invisible/virtual port which does work for you.

This mechanism can be used in Scatter Gather Design Pattern.

Related Articles:

https://msdn.microsoft.com/en-us/library/aa954477.aspx

https://abdulrafaysbiztalk.wordpress.com/tag/self-correlating-ports/

http://geekswithblogs.net/sthomas/archive/2006/02/27/70886.aspx

Code Samples:

http://geekswithblogs.net/nestor/archive/2006/12/21/101772.aspx