BizTalk and AppFabric Caching: Problem or Problem Solver?

by stephen-w-thomas | Jan 10, 2011 | Stephen's BizTalk and Integration Blog

In the recent edition of The BizTalker Newsletter I featured the below article.  Read the article below or join the newsletter to have it delivered to your email.  I welcome and comments. feedback, or thoughts on this subject.

Server AppFabric contains two core pieces – Hosting and Caching. Today, I’m going to focus on the Caching piece of AppFabric.
AppFabric Caching is a new offering from Microsoft as part of the Server AppFabric collection of technologies for the Application Server role inside Windows. AppFabric Caching is Microsoft’s Distributed Cache solution for on-premise, fast, in-memory, and durable object caching.

A lot of community bloggers have been blogging about the use of AppFabric Caching with BizTalk Server. A few of the best posts are:

• Using Windows Server AppFabric Caching for Storing SSO & Configuration data by Abdul Rafay – This shows how to Cache SSO data inside BizTalk.
• Use AppFabric Cache to cache your WCF Service response by Mikael Hakansson – This shows how to Cache WCF response messages inside BizTalk.
• Using AppFabric Cache in BizTalk by Todd Dhaeyer – This shows how to use AppFabric to Cache reference data and even prevent double messages.

Is Server AppFabric Caching really solving problems or creating new ones?

Rather than diving too deep into AppFabric Caching, let me summarize with a list of Do’s and Don’ts.

AppFabric Rules of Thumb – Don’ts:

• Never put anything in the Cache that MUST not be lost – it is a non-transactional cache, not a database.

• Never install AppFabric Caching along with other server products.  It will take up all of the available resources of the server including the entire available RAM so it should only be installed on a dedicated server.

• Avoid making a connection to the Cache Cluster with each request.  The initial connection to the Cache Cluster is more expensive than reading and writing to the Cache. 

AppFabric Rules of Thumb – Do’s:

• Use the Cache for fast, in-memory access to serializable .Net Objects that may or may not actually be in the Cache.

• To achieve fault tolerances in the Cache, simply add a second node to the Cache Cluster (in most cases).

• Use the Cache to reduce the load, performance, and potentially the license costs of more expensive products like SQL Server.

• As demand grows, add more servers to the Cache Cluster to increase the available RAM for Caching.

The guidelines seem simple enough, so what is my hold out on BizTalk and AppFabric Cache?

Using BizTalk and AppFabric together makes awesome demos and can simplify some problems like working with reference data.  The current release of AppFabric Caching is designed to run alone on its own server since it could take up all available RAM.  This would not be a good situation on a production BizTalk Server.

My main concern with AppFabric Caching is if you have a separate caching layer along with your BizTalk Server, why not just use SQL rather than the Cache? 

• Lower Cost – AppFabric Cache is included with the Windows License but you need extra hardware to run it on its own.

• Better Performance – AppFabric Caching would reduce the load on SQL but I would like to see better data on SQL connection vs. Cache connection costs.  Although, if the existing SQL Cluster was already reaching capacity, moving more items away from that would be a plus. 

• Lower Latency – In my mind, reducing latency is the biggest factor for using a Cache.  Cache would reduce the latency vs. a call to the database.  I would say in most BizTalk Solutions an external call to a SQL Server would probably not be the bottle neck given other BizTalk tuning parameters.  

• Mission Critical Data – SQL wins out here (at least for now) because it has a true transactional backend data store. 

I love all three of the AppFabric community demos shown above.  At this stage, I would be comfortable using AppFabric Cache for reference and SSO data – not so much for removing double messages unless I was OK with maybe missing one now and then (but then what’s the point?).  I would still look to leverage SQL for double message removal given the need for true transactions.  Like everything – your unique situation may vary.

In conclusion, BizTalk and AppFabric Caching can work together.  Will caching work in scenarios such as a message re-play and de-duper?  Absolutely!  But for me, given the criticality of most BizTalk installations, I would use caution on using AppFabric Caching with BizTalk at least for now.