How to choose the AS2 Signing Algorithm in BizTalk 2016

How to choose the AS2 Signing Algorithm in BizTalk 2016

For one of our customers we migrated all their interfaces from BizTalk 2006 R2 to BizTalk 2016. During testing of the new BizTalk 2016 environment we found that the signature for the AS2 messages being sent out was not working correctly. While there was no exception in BizTalk, the external party, that was receiving the messages, was unable to verify the signature of the messages. The messages from the old BizTalk 2006 R2 environment were all verified and processed successfully. Obviously we started checking if all of the certificates and party settings were setup correctly in the new BizTalk 2016 environment. We found those to be correct and continued to search for the cause of this issue.

We ended up finding a difference when comparing the signing algorithms. The old BizTalk 2016 R2 environment was using SHA1 while the new BizTalk 2016 machine was using SHA256. Having found this clue, we figured that the fix would be easy: just change the signing algorithm on the AS2 agreement. However, this is where we ran into some problems. It turns out there really isn’t anywhere to configure this on the AS2 agreement. As shown in the picture below, it is possible to specify that the message should be signed, but it is not possible to specify a signing algorithm.

 
The documentation does not specify where to supply the signing algorithm. But after walking through all of the settings of the AS2 agreement again, I noticed that the signing algorithm for the MDN was set to SHA256 and not SHA1. While it is greyed out and, at least according to the screen, only used for MDN’s, we decided to change it anyway and see if this could be the issue.


 
I enabled ‘Request MDN’ and ‘Request signed MDN’ after which I could change the signing algorithm to SHA1. Finally, I disabled ‘Request MDN’ and ‘Request signed MDN’ again since we are not using the MDN.


This finally solved our issue with the signing of the message as now the SHA1 algorithm was used to sign the message!

In conclusion, it is possible to specify the signing algorithm for outgoing messages, but it is not where you would expect it to be. If you interpret the screens of the AS2 party agreement you would think that the signing algorithm can only be specified for MDN’s as it is greyed out by default.

Hopefully the choice of signing algorithm will be easier after a bugfix or in the next release of BizTalk.  

I enabled ‘Request MDN’ and ‘Request signed MDN’ after which I could change the signing algorithm to SHA1. Finally, I disabled ‘Request MDN’ and ‘Request signed MDN’ again since we are not using the MDN.

JSON Encoder Type Bug

JSON Encoder Type Bug

The issue popped up at one or our projects, where we had to deliver a JSON file according to the specifications of an external party. The schema had multiple fields defined as decimal, but for some reason some of the decimals came out as strings. The difference is that the decimal value does not have quotes surrounding the actual value.
To recreate the issue, I created a very simple schema (which is specified below) and a send pipeline containing only the out of the box JSON Encoder.

I’ve chosen to base this scenario on receiving an XML file and sending a JSON file. For this I created a simple messaging-only solution with a file-based Receive Port and file-based Send Port, where the routing is done based on BTS.ReceivePortName. To test this setup I used the following test message.

This is where the issue shows itself. The JSON that is sent by BizTalk is not equal to the expected JSON output. See the comparison and the highlighted difference below.

This is very strange behavior, since both Level1/Field1 and Level1/Field2 are specified as a decimal, and yet Field1 is parsed as a string and Field2 is parsed as a decimal.
The important thing to note is that I have an element called “Field1” on multiple levels in the schema, the first has the type string, the second one has the type decimal.
What appears to be happening is that if you have multiple nodes on different levels in your schema the JSON Encoder always takes the type of the first occurrence of a node with the same name. In our case the first time ”Field1” occurs in our schema it is defined as a string and this is why in our output the second occurrence of the “Field1” node is incorrectly written as a string.
To prove this behavior I renamed the second occurrence of the “Field1” node to “Field3”, this time the output was as expected.

This obviously can be fixed very easily by renaming the fields. However I often find myself in the situation that the XSD cannot be changed as it is defined by an external party. It turns out that the out of the box JSON Encoder uses an old version of the Newtonsoft.Json library which I cannot find in the the Newtonsoft.Json respository on GitHub, so it probably is a Microsoft fork of the Newtonsoft.Json library.

This was all developed and tested on a BizTalk 2016 machine, but I suspect this bug has been present since the introduction of the out of the box JSON Encoder pipeline component with BizTalk 2013R2.

To solve this issue I had to write my own custom JSON Encoder pipeline component where I used the latest version of the Newtonsoft.Json library.

In fact, this issue has been raised to Microsoft via the BizTalk Server uservoice pages. You can find the topic here. If you agree, go there, and show your support by voting for this issue. 

Delivery Notification and the HTTP 500 Status Code

Delivery Notification and the HTTP 500 Status Code

At one of our customers I had implemented the ReturnAddress messaging pattern (http://www.enterpriseintegrationpatterns.com/patterns/messaging/ReturnAddress.html), by using a generic BizTalk application which sends an asynchronous response message to a client application. The solution had been running successfully for some time, when we encountered a strange situation.

The BizTalk application uses a one-way WCF-Custom send port, using a wsHttpBinding, to send a message to the client application. Also, I had added the Delivery Notification functionality to make sure messages are delivered successfully.

It is important to realize that one-way send ports that use SOAP will receive a technical response containing an HTTP status code. If and when the send ports receive a HTTP status code in the 200 range, the Delivery Notification generates an ACK and BizTalk knows the message was successfully delivered.

So far, so good. The application had been through testing on the Test and Acceptance environments, had been deployed to the Production environment and had been running for several months without any problems. Until it appeared that some of the messages that were being sent using the generic BizTalk application were ‘not arriving’ at the client application. This would happen at random and, what was really strange, the logging in BizTalk showed that the message was successfully sent and BizTalk had received an ACK response as part of the Delivery Notification. Also there was no mention of an error in the event log of the BizTalk servers.

After some debugging we found the source of the problem. The message sent by BizTalk was successfully received by the client application, however the client application encountered an error processing the message and returned the HTTP 500 status code. So now the question was, why is the Delivery Notification not generating a NACK response when a HTTP 500 status code is received? I had expected that any status code in the HTTP 400 and 500 range would result in a NACK.

This turned out not to be the case. While status codes in the HTTP 400 range will result in a NACK, the status codes in the HTTP 500 range will result in an ACK and BizTalk will view this message as successfully delivered at the client application. The logic behind this seems to be that the status codes in the HTTP 400 range indicate that the message was not received by the client application (hence the NACK) and the status codes in the HTTP 500 range indicate that the message was received by the client application, but that the client application encountered an exception. Since the message was delivered at the client application, BizTalk views this as a successful delivery and will generate an ACK as part of the Delivery Notification.

Unfortunately, there isn’t any documentation on MSDN on which status codes will return an ACK or NACK.

The documentation on the SOAP HTTP response states that “In case of a SOAP error while processing the request, the SOAP HTTP server MUST issue an HTTP 500 “Internal Server Error” response and include a SOAP message in the response containing a SOAP Fault element indicating the SOAP processing error”. For reference, see https://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383529.

Some discussion followed on the validity of catching the HTTP 500 error in BizTalk, since the message was successfully delivered and accepted by the client application. That means that, from a technical perspective, the responsibility would now lie at the client application to handle the error. From a functional responsibility perspective however, it was decided to find a way to catch the HTTP 500 error in BizTalk, as this would enable the customer’s administrators to use the same resubmit functionality we had created by using a generic BizTalk error handling framework.

So I had to make sure the HTTP 500 status code was somehow caught, so that BizTalk would return a NACK which would result in the error handling catching the error. Fortunately, this can be achieved quite easily by implementing a WCF behavior on the one-way send port. The WCF behavior checks in the AfterReceiveReply message inspector if the reply is a fault message, and if so it will throw an exception using the fault description.

By implementing this WCF behavior on a one-way send port BizTalk will generate a NACK when a response is received with an HTTP status code in the 400 or 500 range. Sometimes the default behavior surrounding technical responsibility doesn’t align with the requirements and responsibilities from a functional point a view, and this may just offer a solution for you as well.