BizTalk Server: Basics principles of Maps – Basic maps functionalities (Document mapping) (Part 4)

BizTalk Server: Basics principles of Maps – Basic maps functionalities (Document mapping) (Part 4)

by | Apr 2, 2012 | BizTalk Community Blogs via Syndication

When we perform a transformation in the message there are 5 basic functionalities that typically arise: Simple mapping of a given value (direct copy) Concatenation of values Conditional selection Custom scripts Add new values (data) Simple mapping of a given value (direct copy) This is the most basic operation of all, where we intend to […]
Blog Post by: Sandro Pereira

Issues with Tracking causing high CPU usage on BizTalk SQL Server (Additional Info)

by | Apr 2, 2012 | BizTalk Community Blogs via Syndication

Since the 1st occurrence of this issue (Issues with Tracking causing high CPU usage on BizTalk SQL Server), I have had 2 other clients with similar issues, each with different root causes. The key finding about these issues is that it is not obvious what the problem is from the initial problem determination, one of the recent issues showed up as a throttling issue for database size, but when running the Message Box Viewer tool, there was no errors that indicated the root cause, but there were warnings around the size of on of the tracking_1_x tables.

The underlying issue with the 2nd occurrence of this issue was a failed upgrade from 2006 R2 to 2009, although the upgrade seemed to work correctly, the user running the upgrade did not have sysadmin privileges on the BizTalk SQL Server, as required in the install instructions. This lack of permission lead to the service account for the Tracking host not being given permission on the tables and stored procedures that are used doing the movement of tracking data from the message box database to the tracking database. Once the permissions were update (cross checked permissions from a successfully upgraded 2009 BizTalk SQL Server), tracking data started to be moved from the message box database to the tracking database.

The 3rd occurrence is still under investigation, but the symptoms are the same, High CPU on the BizTalk server and large number of records in the tracking_1_x tables in the message box database.

From now on, one of my 1st checks for a poorly performing BizTalk system will be check the tracking_1_x tables and the TDDS_StreamStatus table. I will be publishing a SCOM monitor to detect the tracking data not being copied and hopefully a rule for message box viewer that will clearly identify this issue.

Transactional Messaging in the Windows Azure Service Bus

by | Apr 2, 2012 | BizTalk Community Blogs via Syndication

Introduction

I’m currently working on broadening the content in the Windows Azure Service Bus Developer Guide. One of the features I have been looking at over the past week is the support for transactional messaging. When using the direct programming model and the WCF interface some, but not all, messaging operations can participate in transactions. This allows developers to improve the reliability of messaging systems. There are some limitations in the transactional model, transactions can only include one top level messaging entity (such as a queue or topic, subscriptions are no top level entities), and transactions cannot include other systems, such as databases.

As the transaction model is currently not well documented I have had to figure out how things work through experimentation, with some help from the development team to confirm any questions I had. Hopefully I’ve got the content mostly correct, I will update the content in the e-book if I find any errors or improvements that can be made (any feedback would be very welcome). I’ve not had a chance to look into the code for transactions and asynchronous operations, maybe that would make a nice challenge lab for my Windows Azure Service Bus course.

Transactional Messaging

Messaging entities in the Windows Azure Service Bus provide support for participation in transactions. This allows developers to perform several messaging operations within a transactional scope, and ensure that all the actions are committed or, if there is a failure, none of the actions are committed. There are a number of scenarios where the use of transactions can increase the reliability of messaging systems.

Using TransactionScope

In .NET the TransactionScope class can be used to perform a series of actions in a transaction. The using declaration is typically used de define the scope of the transaction. Any transactional operations that are contained within the scope can be committed by calling the Complete method. If the Complete method is not called, any transactional methods in the scope will not commit.

// Create a transactional scope.

using (TransactionScope scope = new TransactionScope())

{

// Do something.

// Do something else.

// Commit the transaction.

scope.Complete();

}

In order for methods to participate in the transaction, they must provide support for transactional operations. Database and message queue operations typically provide support for transactions.

Transactions in Brokered Messaging

Transaction support in Service Bus Brokered Messaging allows message operations to be performed within a transactional scope; however there are some limitations around what operations can be performed within the transaction.

In the current release, only one top level messaging entity, such as a queue or topic can participate in a transaction, and the transaction cannot include any other transaction resource managers, making transactions spanning a messaging entity and a database not possible.

When sending messages, the send operations can participate in a transaction allowing multiple messages to be sent within a transactional scope. This allows for “all or nothing” delivery of a series of messages to a single queue or topic.

When receiving messages, messages that are received in the peek-lock receive mode can be completed, deadlettered or deferred within a transactional scope. In the current release the Abandon method will not participate in a transaction. The same restrictions of only one top level messaging entity applies here, so the Complete method can be called transitionally on messages received from the same queue, or messages received from one or more subscriptions in the same topic.

Sending Multiple Messages in a Transaction

A transactional scope can be used to send multiple messages to a queue or topic. This will ensure that all the messages will be enqueued or, if the transaction fails to commit, no messages will be enqueued.

An example of the code used to send 10 messages to a queue as a single transaction from a console application is shown below.

QueueClient queueClient = messagingFactory.CreateQueueClient(Queue1);

Console.Write(“Sending”);

// Create a transaction scope.

using (TransactionScope scope = new TransactionScope())

{

for (int i = 0; i < 10; i++)

{

// Send a message

BrokeredMessage msg = new BrokeredMessage(“Message: “ + i);

queueClient.Send(msg);

Console.Write(“.”);

}

Console.WriteLine(“Done!”);

Console.WriteLine();

// Should we commit the transaction?

Console.WriteLine(“Commit send 10 messages? (yes or no)”);

string reply = Console.ReadLine();

if (reply.ToLower().Equals(“yes”))

{

// Commit the transaction.

scope.Complete();

}

}

Console.WriteLine();

messagingFactory.Close();

The transaction scope is used to wrap the sending of 10 messages. Once the messages have been sent the user has the option to either commit the transaction or abandon the transaction. If the user enters “yes”, the Complete method is called on the scope, which will commit the transaction and result in the messages being enqueued. If the user enters anything other than “yes”, the transaction will not commit, and the messages will not be enqueued.

Receiving Multiple Messages in a Transaction

The receiving of multiple messages is another scenario where the use of transactions can improve reliability. When receiving a group of messages that are related together, maybe in the same message session, it is possible to receive the messages in the peek-lock receive mode, and then complete, defer, or deadletter the messages in one transaction. (In the current version of Service Bus, abandon is not transactional.)

The following code shows how this can be achieved.

using (TransactionScope scope = new TransactionScope())

{

while (true)

{

// Receive a message.

BrokeredMessage msg = q1Client.Receive(TimeSpan.FromSeconds(1));

if (msg != null)

{

// Wrote message body and complete message.

string text = msg.GetBody<string>();

Console.WriteLine(“Received: “ + text);

msg.Complete();

}

else

{

break;

}

}

Console.WriteLine();

// Should we commit?

Console.WriteLine(“Commit receive? (yes or no)”);

string reply = Console.ReadLine();

if (reply.ToLower().Equals(“yes”))

{

// Commit the transaction.

scope.Complete();

}

Console.WriteLine();

}

Note that if there are a large number of messages to be received, there will be a chance that the transaction may time out before it can be committed. It is possible to specify a longer timeout when the transaction is created, but It may be better to receive and commit smaller amounts of messages within the transaction.

It is also possible to complete, defer, or deadletter messages received from more than one subscription, as long as all the subscriptions are contained in the same topic. As subscriptions are not top level messaging entities this scenarios will work.

The following code shows how this can be achieved.

try

{

using (TransactionScope scope = new TransactionScope())

{

// Receive one message from each subscription.

BrokeredMessage msg1 = subscriptionClient1.Receive();

BrokeredMessage msg2 = subscriptionClient2.Receive();

// Complete the message receives.

msg1.Complete();

msg2.Complete();

Console.WriteLine(“Msg1: “ + msg1.GetBody<string>());

Console.WriteLine(“Msg2: “ + msg2.GetBody<string>());

// Commit the transaction.

scope.Complete();

}

}

catch (Exception ex)

{

Console.WriteLine(ex.Message);

}

Unsupported Scenarios

The restriction of only one top level messaging entity being able to participate in a transaction makes some useful scenarios unsupported. As the Windows Azure Service Bus is under continuous development and new releases are expected to be frequent it is possible that this restriction may not be present in future releases.

The first is the scenario where messages are to be routed to two different systems.

The following code attempts to do this.

try

{

// Create a transaction scope.

using (TransactionScope scope = new TransactionScope())

{

BrokeredMessage msg1 = new BrokeredMessage(“Message1”);

BrokeredMessage msg2 = new BrokeredMessage(“Message2”);

// Send a message to Queue1

Console.WriteLine(“Sending Message1”);

queue1Client.Send(msg1);

// Send a message to Queue2

Console.WriteLine(“Sending Message2”);

queue2Client.Send(msg2);

// Commit the transaction.

Console.WriteLine(“Committing transaction…”);

scope.Complete();

}

}

catch (Exception ex)

{

Console.WriteLine(ex.Message);

}

The results of running the code are shown below.

When attempting to send a message to the second queue the following exception is thrown:

No active Transaction was found for ID ’35ad2495-ee8a-4956-bbad-eb4fedf4a96e:1′. The Transaction may have timed out or attempted to span multiple top-level entities such as Queue or Topic. The server Transaction timeout is: 00:01:00..TrackingId:947b8c4b-7754-4044-b91b-4a959c3f9192_3_3,TimeStamp:3/29/2012 7:47:32 AM.

Another scenario where transactional support could be useful is when forwarding messages from one queue to another queue. This would also involve more than one top level messaging entity, and is therefore not supported.

Another scenario that developers may wish to implement is performing transactions across messaging entities and other transactional systems, such as an on-premise database. In the current release this is not supported.

Workarounds for Unsupported Scenarios

There are some techniques that developers can use to work around the one top level entity limitation of transactions. When sending two messages to two systems, topics and subscriptions can be used. If the same message is to be sent to two destinations then the subscriptions would have the default subscriptions, and the client would only send one message. If two different messages are to be sent, then filters on the subscriptions can route the messages to the appropriate destination. The client can then send the two messages to the topic in the same transaction.

In scenarios where a message needs to be received and then forwarded to another system within the same transaction topics and subscriptions can also be used. A message can be received from a subscription, and then sent to a topic within the same transaction. As a topic is a top level messaging entity, and a subscription is not, this scenario will work.

BizTalk Server: Basics principles of Maps – Introduction to map editor (Part 3)

BizTalk Server: Basics principles of Maps – Introduction to map editor (Part 3)

by | Apr 1, 2012 | BizTalk Community Blogs via Syndication

The map editor, BizTalk Mapper Designer, enables us to perform transformations of complex messages in a visual and extremely simple way, expressed in graphics associations of links that define the relationships between the various elements of messages. These relationships between elements are internally implemented as XSL Transformations (XSLT – Extensible Stylesheet Language Transformation) which is […]
Blog Post by: Sandro Pereira

Publishing an InfoPath form to Sharepoint 2010 for WebBrowser

by | Apr 1, 2012 | BizTalk Community Blogs via Syndication

Publishing an InfoPath form to SharePoint 2010 is not that difficult, that is difficult is getting it published, when it has code behind for the use in the Web Browser.

I recently battled with this, and I must say it’s better than it used to be, making InfoPath VERY powerful, particularly if you can use code behind, and even call your own .net dll’s from the form.

For most they have got somewhere, but have not got it fully working.. if you are one of these people read on.

Do you want to activate it in the site collection, but can’t upload it?

First you need to do this:

1. Open SharePoint 2010 Central Administration.
2. Click General Application Settings.
3. On the General Application Settings page under InfoPath Forms Services, click Upload form template.

You GET: ERROR: This form template has not been published. Open the form in the InfoPath Designer and publish the form to SharePoint using the Administrator-approved form template method in the Publishing Wizard.

Do the following:

Important: Ensure that the security and trust is set to full trust, with a certificate.


1. Publish the form to SharePoint from InfoPath.

2. Note down where you published it: http://msite/template.xsn

3. Go to your web browser and go to this URL, download the PUBLISHED template.xsn

4. SAVE IT locally.

OR

4a. Better way: Publish it to a network location say c:\MyFormTemplates, you do not need it in SharePoint YET.

5. Now upload the {template}.xsn, (Rename if you like) to SharePoint admin.

You should get: The form template has been successfully uploaded to the farm. To make the form template available in a site collection, activate the form template from the Manage Form Templates page or from the feature activation page in the site collection.

6. Now you can activate it in your site collection… etc…

Go to your site where you want to use the form.

Site settings / site collection administration

Site collection features

Look for your Infopath Form Template, (it’s sorted alphabetically)

Press Activate

THE KEY to enabling the form for the browser is in how you set up for form library…

Under Site/{FormLibraryName}/form library settings/ advanced settings.

SAY Allow management of content types = YES

Then hit ok.

Now go down to content types, and defined your content type, say add from existing site content types, which you just activated… You do remember the NAME of the form you uploaded in admin before??

Make sure that it is the default content type, and everything will be good.

Grant permissions to your users, and have them press new document, and up comes your form in the browser…

You can then put your workflow on the form library, or hook BizTalk up to the form Library, and you have full end to end Form and Workflow.

.NET: Microsoft’s new .NET language ’D-flat’ .net

by | Apr 1, 2012 | BizTalk Community Blogs via Syndication

Local MS Developer pillar Andrew Coates spilled the beans on this next new language
to come out of MS Research.

Db.NET or ’D flat’ – F#, C# and the Cinderella of the 3 sisters ’VB.NET’

(Last year I was introduced to F# over a 5 month project and absolutely loved the
simplicity and freshness of it – async was simple, tasks, functions and code that
would normally take 400 lines in C#, we were able to do in 100 in F#)

It promises:

– speed

– optimisation (I wonder if it’ll be smart enough to run tasks on different CPU cores?)

There is a focus on Orchestration – data Orchestration found here http://thenextlanguage.net/a-focus-on-orchestration/

Where it talks about “An example of the close collaboration between the product team
and the company’s research arm is the use of Schenkerian Analysis in
the compiler to maximize orchestration between sections of the code.”

Oooh I thought – let’s check out what this is Schenkerian Analysis and
a quick check of Wikipedia reveals http://en.wikipedia.org/wiki/Schenkerian_analysis

“Schenkerian analysis is a method of musical
analysis of tonal music based on the theories of Heinrich
Schenker. The goal of a Schenkerian analysis is to interpret the underlying structure
of a tonal work. The theory’s basic tenets can be viewed as a way of defining tonality in
music. A Schenkerian analysis of a passage of music shows hierarchical relationships
among its pitches, and draws conclusions about the structure of the passage from this
hierarchy. The analysis is demonstrated through reductions of the music, using a specialized
symbolic form of musical notation that Schenker devised to demonstrate various prolongational
techniques. The concept of tonal prolongation, in which certain pitches determine
the goal of other, subordinate pitches, is a cornerstone of the pitch hierarchy that
Schenkerian analysis involves itself with.”

So tones, pitches and music is where this algorithm has its rootsI can see how you
could take this analysis when applied to the frequency of music and apply it to the
frequency of code items; data being hit etc.

I’ll crack open this VS.2011 extension and see what transpires

Grab the TOOLS here – http://thenextlanguage.net/tools/

Blog Post by: Mick Badran

I’m going to Amsterdam

by | Mar 30, 2012 | BizTalk Community Blogs via Syndication

Today I registered for TechEd Europe 2012. Last time was in cold autumn Berlin. This time it will be warm summer Amsterdam. Time to stock up on motivation, knowledge, contacts and also some “Bitter Cookies” and Heineken.

Last time I blogged my proverbial a$$ off and I can guarantee I will do the same this time.

Blog Post by: Mikael Sand