BizTalk Host setup for your First BizTalk Projects

Ive been speaking to a few people starting their organisations first BizTalk projects recently and talking about Host setup and what is a good intial setup and how it may change over time.

Normally you see companies go one of two ways to begin with:

1. They just have one BizTalk host with an instance on each server and use it for everything

2. They have many instances for everything and use way more than they need to.

Im a big believer that a little thought about this and how your company plans to use BizTalk over time can help ensure you dont end up messing around with your host setup constantly.

I was saying to the team that probably the best way to start is create the following hosts to begin with.

  • A host called GenericRecevice which would be a host that your in process receive adapters can use
  • A host called GenericApplication which is a host for all of your orchestrations
  • A host called GenericSend which is a host for all of your send ports
  • A host called Tracking which is used for nothing but tracking.

I recommended they left the BizTalkApplication host because it can be the default incase anyone doesnt follow the above rules but do not have any instances of this host on any servers so your team are forced to fix configuration mistakes and to follow your host rules.

When you setup your host instances this obviously varies where you put each host instance based on the topology of your BizTalk group but having already splitt out your artefacts like suggested abovemakes it easier to move your hosts around in a consistent manner.

At this point you not have a clear well understood host setup and some simple rules around when you would use each one, however the key thing is that there will often be potential exceptions to these rules and you should ensure that if you want to create a new host for one or more artefacts you are able to explain clearly the reason for this new host and not just have a new host because the developer used a different name like Ive seen happen before.

Lastly a thought on the isolated host. We often tend to just have the one BizTalk Server Isolated Host which comes out of the box and run different adapter typeson seperate app pools in IIS as an example. This will cover the majority of cases but of course again there will be occasions where you have a need to do something different which is fine so long as its clearly explained.

I would say that generally good reasons for new hosts out side of this patten will be:

  • Performance reasons
  • Security Isolation
  • Adapter specific scenarios

Just a final note that this is intended as some advice when you start your companies first BizTalk projects. If you set of with this configuration it should cover you nicelyfor most of your cases.

Windows Azure tables: partitioning guid row keys | Windows Azure tables: partionnement de clefs de type GUID

 

 

This post is about using a partition key while the rowkey is a GUID in Windows Azure Table Storage. You may read more on Windows Azure Tables partitioning in this MSDN article. Ce billet est %u00e0 propos de l’utilisation d’une clef de partition alors que la clef est un GUID lors de l’utilisation des tables non relationnelles de Windows Azure. On trouvera des informations compl%u00e9mentaires sur le partitionnement des tables Windows Azure dans cet article MSDN.
Here is some context on when this sample code comes from. Voici un peu de contexte sur l’origine de cet exemple de code.
I’m currently developing some sample code that works as a canvas application in Facebook. As this type of app runs in a iFrame, it restricts the use of cookies. Still, I wanted to keep some authentication token from page to page (the Facebook signed_request) without displaying it explicitly in the URL (it may also be sent to HTTP referer headers to external sites). So i decided to store the signed_request in a session. ASP.NET sessions can have their ID stored in the URL instead of cookies but ASP.NET pipelines does not provide the session yet while authenticating. So I created my own storage for the authentication and authorization session (Auth Session for short). I did it in Windows Azure tables so that it can easily scale out. Je d%u00e9veloppe actuellement un exemple d’application Facebook de type Canvas. Comme ces types d’applications s’ex%u00e9cutent dans une IFrame, les cookies sont difficilement utilisables. Cependant je devais tout de m%u00eame pouvoir garder un jeton d’authentification de page en page (le champ signed_request de Facebook) sans l’afficher explicitement dans l’URL (il pourrait entre autres %u00eatre envoy%u00e9 dans l’ent%u00eate referer HTTP vers des sites externes). J’ai donc choisi de stocker le signed_request dans la session. ASP.NET propose de stocker l’ID de session dans l’URL plut%u00f4t que dans des cookies mais l’ordre des %u00e9v%u00e9nements ASP.NET fait que lors de l’authentification la session n’est pas encore disponible. J’ai donc cr%u00e9%u00e9 mon propre stockage pour la session d’authentification et d’autorisation (en abr%u00e9g%u00e9: Auth Session). Je l’ai fait en m’appuyant sur les tables Windows Azure de fa%u00e7on %u00e0 b%u00e9n%u00e9ficier simplement de la mont%u00e9e en charge horizontale.
The functional key is a GUID (I don’t want user X to guess user Y’s authSessionKey). The key is passed from page to page as a query parameter (typically, app urls look like this: https://myhost/somepath?authSessionKey=3479D7A2-5D1A-41A8-B8FF-4F62EB1A07BB. La clef fonctionnelle est un GUID (je ne veux pas que l’utilisateur X puisse deviner la clef de session d’un utilisateur Y). Cette clef est pass%u00e9e de page en page sous la forme d’un param%u00e8tre de la requ%u00eate (typiquement les urls de l’app. ressemblent %u00e0 ceci: https://myhost/somepath?authSessionKey=3479D7A2-5D1A-41A8-B8FF-4F62EB1A07BB.
Still, in order to have this scaling horizontally I need to have a partition key. Here is the code I used: Il reste que pour pouvoir monter en charge horizontalement, il faut une clef de partition. Voici le code:

  

internal class AuthSessionDataSource
{
//...
        public const int nbPartitions = 15;
// ...

public static class AuthSessionState
{
//...
    private static string PartitionKeyForGuid(Guid guid)
    {
        int sumOfBytes = 0;
        foreach (var x in guid.ToByteArray())
        {
            sumOfBytes += x;
        }
        int partitionNumber = sumOfBytes % AuthSessionDataSource.nbPartitions;
        return partitionNumber.ToString();
    }
//...
The principle is to get the remainder of the sum of all bytes participating in the GUID divided by the number of partitions as the partition number. Le principe est de prendre comme num%u00e9ro de partition le reste de la division de la somme des octets qui consituent le GUID par le nombre de partitions.
In order to have a rough idea of what it provides, here is a small console application (code, then sample execution result). De fa%u00e7on %u00e0 avoir une id%u00e9e rapide de ce que cela donne, on peut mettre le code dans une application console et l’ex%u00e9cuter.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication2
{
    class Program
    {
        static void Main(string[] args)
        {
            for (int i = 0; i < 36; i++)
            {
                Guid g = Guid.NewGuid();
                Console.WriteLine("{0}, {1}", g, PartitionKeyForGuid(g));
            }
            Console.ReadLine();


        }


        private static string PartitionKeyForGuid(Guid guid)
        {
            const int nbPartitions = 12;

            int sumOfBytes = 0;
            foreach (var x in guid.ToByteArray())
            {
                sumOfBytes += x;
            }
            int partitionNumber = sumOfBytes % nbPartitions;
            return partitionNumber.ToString();
        }

    }
}

 

The advantage is that the partition numbers should be distributed quite regularly and that you can get calculate the partition from the rowkey as long as the number of partitions doesn’t change. L’avantage est que les num%u00e9ros de partition devraient %u00eatre distribu%u00e9s assez r%u00e9guli%u00e8rement et que l’on peut calculer ce num%u00e9ro de partition %u00e0 partir de la clef (le GUID) tant que le nombre de partitions ne change pas.
Should I change and have more partitions as the number of users grow, I could store new users’ sessions to a new table where the number of partitions is higher while keeping already active users to the old table. Auth sessions don’t live very long so changing the number of partitions can be quite simple. Et si on devait changer le nombre de partitions parce que le nombre d’utilisateurs augmente? Dans mon cas, je peux stocker les session des nouveaux utilisateurs dans une nouvelle table avec un plus grand nombre de partitions tout en gardant les utilisateurs d%u00e9j%u00e0 actifs dans l’ancienne table. Les Auth Sessions n’ont de toutes fa%u00e7ons pas une dur%u00e9e de vie tr%u00e8s longue et changer le nombre de partitions devrait %u00eatre assez simple.

 

public const int nbDaysForOldSessions = 3;
//...
internal void RemoveOldValues()
{
    DateTime oldDate = DateTime.UtcNow.AddDays(-1 * nbDaysForOldSessions);

    for(int p=0; p<nbPartitions; p++)
    {
        string partitionKey = p.ToString();
        var query = (from c in context.AuthSessionStateTable
                     where c.PartitionKey == partitionKey
                     && c.Timestamp <= oldDate
                     select c)
                    .AsTableServiceQuery<AuthSessionStateEntity>();
        var result = query.Execute();
        int i = 0;
        foreach (var x in result)
        {
            i++;
            this.context.DeleteObject(x);
            if (i >= 100)
            {
                this.context.SaveChangesWithRetries();
                i = 0;
            }
        }
        this.context.SaveChangesWithRetries();
    }
}

 

Why not using the rowkey as a partition key? Well having several rows in the same partition allows batching which is also good for performance. For instance, I have to remove old sessions. As batch can only happen in a same partition and as no more than 100 rows can be batched together, here is the code to purge old Auth sessions: Pourquoi ne pas utiliser la clef (le GUID) en tant que clef de partition %u00e9galement? En fait, avoir plusieurs rang%u00e9es dans la m%u00eame partition permet de grouper des requ%u00eates ce qui am%u00e9liore aussi les performances. Par exemple, j’ai besoin de supprimer les vieilles sessions. Comme les regroupements ne peuvent se faire que dans la m%u00eame partition et que 100 enregistrements par 100 enregistrements voici le code de purge des vieilles sessions:
In my case, having this way of partitioning data, seems to be a good fit. Dans mon cas, partitionner les donn%u00e9es de cette fa%u00e7on semble assez adapt%u00e9.

 

Benjamin

Blog Post by: Benjamin GUINEBERTIERE

BizTalk Server 2010 Cookbook by Steef-Jan Wiggers

BizTalk Server 2010 Cookbook by Steef-Jan Wiggers

In my previous post  I promised that my next blog post would be a follow up on the CRM2011 integration example.

Covering a rewrite of the CRM Workflow Component such that it incorporates functionality which uses the Windows Azure Service Bus Relay Services to send over the CRM data to BizTalk.

Well something kind of, came in between.. But no worries, I am still working on putting it all down in a blogpost and I hope to publish it pretty soon.

But in the meantime, keep on reading,  relax, get your mouse or your finger-tips ready to click or swipe a-long as I shamelessly present another blog-post dedicated to another great BizTalk Server book which is a must read…

It has been almost 2 months, since Packt publishing released Steef-Jan Wiggers book “BizTalk Server 2010 Cookbook“.

I had the privilege to review this book together with fellow BizTalk-ers; Randal van Splunteren, Abdul Rafay and Sandro Pereira

Steef-Jan put in a lot of effort writing this book, and well this book deserves all the attention it can get! So that’s the reason  I’ve decided to shamelessly plug it.

Like I said, it has been almost 2 months since Packt released this book and I for once could write yet another elaborate review explaining that Steef-Jan’s book is:

unique in its kind, must have, must read, no-nonsense,easy to read, covering almost all aspects ranging from development, architecture, testing to deployment, targets not only developers, architects, administrators and people who don’t work with BizTalk each and every day etc..

however by doing so I would merely repeat what a lot of fellow BizTalk-ers have been blogging about, so without further a due I present a small collection of links  pointing to reviews of Steef-Jan Wiggers book “BizTalk Server 2010 Cookbook”

Reviews by (in no particular order)

Tord Glad Nordahl

Lex Hegt

Mikael Sand
Saravana Kumar
Howard S. Edidin

Sandro Pereira

Antonino Crudele

Stephen W. Thomas
Abdul Rafay

Sandro Pereira

Well what’s keeping you here? Click on one of the links to buy this great book.

Cheers and till next time

René

BizTalk Server 2010 Cookbook Give-A-Way

I have known Steef-Jan Wiggers for many years now. He is a fellow BizTalk Server MVP and one of the sharpest BizTalk guys I know. So of course I was excited when I found out he was working on a solo BizTalk book.

His book is a cookbook-style book named BizTalk Server 2010 Cookbook by Packt Publishing. I like this style of book because it contains smaller units of examples related to a specific area or to solve a specific problem.

This book covers areas like setup, patterns, error handling, AppFabric connect, administration, and testing.  They are all easy to follow and it contains great illustrated examples.

This book is available on Amazon (currently with an awesome Kindle price of under $26) and online at Packt.com.

Packt is currently running some great specials at the Packt Microsoft Carnival.

While combing through some blogs today, I found two recent blog posts related to the book I co-authored, Applied Architecture Patterns on the Microsoft Platform. One was by Buck Woody and the other by Mark Brimble. It is great to see great comments on our book almost 2 years after it was published. With Packt’s Microsoft Sale, now is a great time to pick up our book along with the BizTalk Cookbook.

Here is a little known fact – I did not write any of the BizTalk content for our book – I did all of the AppFabric, Service Bus, and SharePoint sections!  Um, you all probably already knew that from the poor grammar and I am sure some spelling errors.

Here is your chance to win a copy of Steef’s new book. As runner-up prizes, I have copies of Dan Rosanova’s BizTalk 2010 Patterns and Pro BAM in BizTalk 2009. This will be a Facebook & Twitter only contest that will start next week.

http://www.facebook.com/BizTalkGurus http://twitter.com/BizTalkGurus

To get the details next week just like us on Facebook or Follow us on Twitter.

Best Missing BizTalk Blog Contest Winners

First off, thanks for all the great suggestions of missing blogs from the BizTalkGurus.com Blog Feed!

I will be working on getting those site included into the feed with the next release that should happen in the next week.

Now for the winners!

Grand Prize of a copy of Microsoft BizTalk Server 2010 Patters book by Dan Rosanova goes to Mark Brimble.

Runner up prize of a copy of Pro Business Activity Monitoring in BizTalk 2009 signed by BOTH authors goes to Stuart Brierley.

Thanks again for all the great feedback and look for more books give-a-ways in the next few days.

BizTalk Innovation Event, Milan, Italy | May 24th

Overnet Education in collaboration with Microsoft Italy and User Group Italiano Connected System has organaised one of the biggest BizTalk event in Italy.

All thanks goes to Nino Crudele (fellow Italian BizTalk Server MVP) for taking the effort to organise the event. It’s not a easy task by no means organising a community activity (FREE) of this magnitude bringing international speakers. Big thanks to Nino for organising this.

If you are somewhere near by (Sweden, Norway, UK or any European destination), it will be a worthy while day to network with some of the great guys in BizTalk community.

The agenda for the event:

BizTalk Host thresholds and automatic throttling
by Tord Glad Nordahl;

BizTalk360
by Saravana Kumar (MVP BizTalk Server)

Adapter Pack Integration Capabilities (BizTalk)
by Steef-Jan Wiggers (MVP BizTalk Server)

Introduction to the Azure Service Bus EAI/EDI features
by Sandro Pereira (MVP BizTalk Server)

BizTalk Innovation Event Wrap up
by Nino Crudele (MVP BizTalk Server)

Click here to ensure your registration or to get more information?s about this event.

Nandri!

Saravana Kumar

BizTalk Message Based Routing: why is my file not transfered to the location of the Send Port

Inspired by some tweets I had with Sam Vanhoutte (@SamVanhoutte), Mikael Sand (@MikaelSand) and Dean Robertson (@DeanRobertson), I decided to write down a number of our experiences of what can go wrong, or what you could forget, in case of a simple message based routing scenario.

So basically this article contains a checklist, which can be used as a reminder in case your message based routing scenario does not work, like you expected.

Scenario
Imagine you have a message based routing solution in its most simple form.
Your Receive Location listens to a local folder and your Send Port is subscribed to the Receive Port from the Receive Location. The URI of the Send Port is another local folder.
Although this is a very simple solution, still there is enough that can prevent the messages from flowing from A to B.

1. The Receive Location listens to the wrong folder or contains a typo
To be absolutely sure you have configured the correct folder, copy the path from the address bar in Windows Explorer and paste it at the URI field in the Receive Location.

2. The Receive Location is not enabled
If the Receive Location is not enabled, it won’t poll the configured folder, so you must enable the Receive Location.

3. BizTalk is not authorized to access the file location of the Receive Location
Make sure that the identity which is configured at the Host which runs the Receive Location, has enough permissions (read/write) on the folder of the Receive Location. Failing to do so, results in the Receive Location disabling itself (after some retries).

4. The Host Instance which runs the Receive Location is not started
Host Instances take care of the actual processing of your Receive Locations, Send Ports and Orchestrations. So if the Host Instance from the Receive Location is not started, no files will be picked up, although your Receive Location is enabled, so start the designated Host Instance.

5. The filter on the Send Port contains the wrong Receive Port or contains a typo
Correct this by copying the name from the Receive Port and paste it in the filter of the Send Port.

6. The name of the Receive Port in the filter of the Send Port is surrounded with quotes
Quotes are not needed, so remove them.

7. The Send Port is not enlisted and started
If the Send Port is not enlisted, there is no active subscription for the Send Port. Further the Send Port must be started to send any files, coming from the Receive Port to the outgoing folder. So Enlist and Start the Send Port.

8. The Host Instance which runs the Send Port is not started
Host Instances take care of the actual processing of your Receive Locations, Send Ports and Orchestrations. So if the Host Instance from the Send Port is not started, the file will be picked up by the Receive Location, stay in the MessageBox, but won’t be delivered to the outgoing folder, so start the designated Host Instance.

9. BizTalk is not authorized to access the file location from the Send Port
Make sure that the identity which is configured at the Host which runs the Send Port, has enough permissions (read/write) on the folder of the Send Port. Failing to do so, will result in suspended messages.

10. Mismatch of the file mask on the Receive Location
The file that you drop on the folder of the Receive Location does not match the file mask from the Receive Location. If you are dropping a .XML file while the Receive Location expects only *.TXT files, your .XML file won’t be picked up by the Receive Location.

11. Tricked by Windows Explorer
You think you are dropping a .XML file, and the file mask expects .XML files, but still the file is not picked up. Perhaps you are tricked by Windows Explorer. Windows Explorer has an option to hide familiar extensions of files. So if you have a file called order.xml.txt, Windows Explorer hides the .txt extension, perhaps giving you the idea that you are dealing with a .XML file, while in fact you are dealing with a .TXT file. In Windows Explorer turn off the option to hide familiar extensions, making sure that you will always deal with the full file name.