Leveraging MSMQT acknowledgments

You all probably know that MSMQ stands for ‘Microsoft Message Queuing’ and MSMQT is the acronym for describing the BizTalk MSMQ adapter. A quick reminder for those who already forgot: the “T” in MSMQT stands for transactional (not “Transport”)

Probably less common knowledge is what MSMQ acknowledgments (ACKs) are, so I have chosen this to be the topic for my first post. What are ACKS and in what flavors do they come?

Acknowledgments are system-generated confirmation messages that are sent to the administration queues specified by the sending application. When an application sends a message, it can request that Message Queuing return acknowledgment messages indicating the success or failure of the original message.

Both acknowledgment types have positive (ACK) and negative (NACK) variants indicating – this is too straightforward – a success or a failure: positive arrival acknowledgments, positive read acknowledgments, negative arrival acknowledgments, and negative read acknowledgments.

If you want to get rapidly acquainted with acknowledgments you should definitely read this excellent article on MSDN: Reliable Messaging with MSMQ and .NET (Building Distributed Applications)

Personally having no prior experience with MSMQ acknowledgments, I started my first tests having big expectations. I built a .NET application and sent my 1st MSMQ message with an ACK-level set to full_reach and full_receive using a non-transactional admin queue. (my machine is a simple MSMQ independent client so I send to other machines using outgoing queues). Also not wanting to be too haughty I targeted my first message send at a regular MSMQ server. As expected my message arrived quickly at its destination queue where I subsequently could read the message through code.

Wonderful, I had 2 ACKs in my sending machine’s admin queue:

Since this first test being a big success I already wondered what would happen if I’d send the same message again, but now targeted to the BizTalk MSMQT adapter.

I quickly set up a BizTalk MSMQT receive location with a passthru pipeline and a file send port subscribing to the MSMQT port. After BizTalk created the flat output message I was really surprised – even a bit disappointed at the same time – because I only found 1 acknowledgment in my admin queue:

Where the heck did the receive ack go? Did I make a mistake? I did have a valid subscription and it was consumed correctly because I had a file. After doing some research, newsgroup postings and e-mailing, a very helpful person from MS gave me the answer: ‘For Biztalk, multiple “applications” can receive the message and MSMQT doesn’t know how many got it, so it was decided that receive ACKs would only confuse the customers.’

Slowly getting over my depression I started focusing on the ACK I did receive. I started to think about the opportunities this ACK creates: you now have a level of traceability at the sender’s side. For my next test I sent a message to an invalid MSMQT queue – more precisely I used an invalid name being a non existing queue on MSMQT. Admin queue:

Exactly what I expected! This reminded me of some interesting must-read posts on this subject by Darren Jefford’s blog titled MSMQT, Transactional Queues and Xml Validating Pipelines and MSMQT: Debugging and Getting hold of malformed messages.

Conclusion: if the sending of a message fails the sender receives a NACK message in the admin queue and eventually after exhaustion of the retries MSMQ places the bad message from the outgoing queue in the dead-letter queue (optional: see below for more information). Darren Jefford advices to always use passthru pipelines for MSMQT receive locations. Now let’s ignore his advice and try the opposite. What would happen if I would send a message to an existing queue but an exception is raised from inside the receive pipeline?

I quickly configured my MSMQT receive location to use a pipeline implementing the flat file parse (FFDASM) component. Next I intentionally sent a malformed FF message to the MSMQT queue – generating a parsing error. Here is what I got in my admin queue:

I expected to have a more meaningful description as for example ‘Parsing failed’, but I received the same – now totally irrelevant – description. As a consequence the message sender cannot see the difference between these 2 different types of errors. I asked my MS contact again and he gave me this explanation: ‘because we cannot define new values for MSMQMessage.Class, we were pretty much stuck with the existing ones.’ This means, I think, that they were stuck with the standard MSMQ API error enumerators.

So here’s my conclusion: If you send messages to MSQMT receive locations you should always choose between

(A) Use a passthru pipeline on the receive location or

(B) Use a custom pipeline but specify a dead-letter queue for the message and/or implement acknowledgments/logic to handle failed messages due to pipeline execution errors.

As for the sake of completeness I also want to mention that MSMQT also sends positive ACKS for each successful message receival. This assures us that your message was handled fine and survived pipeline execution without any error.

Now having all these extra bits of information regarding ACKS I’d like to finish by issuing a little warning. An example of a bad design would be to create some kind of service that sends messages 1 by 1 to a MSMQT node – while waiting in between each send for an acknowledgment. This would probably work from a functionality point of view but never from a performance perspective. Read this: ‘Using the Right Tool for the Right Job’ chapter

To finish my article I’d like to comment on the overall very fine transcript of the Microsoft Support WebCast – BizTalk Message Queuing in Microsoft BizTalk Server 2004 where Rajan Dwivedi states ‘As far as interaction between MSMQT and MSMQ is concerned, MSMQT can receive messages from MSMQ, and it can also send messages to MSMQ without any difference.’ Well Rajan, I’m not so sure 😉