Service Virtualization with the Microsoft Managed Services Engine

I presented at the Twin Cities Connected Systems User Group last night on the Managed Services Engine. You can get the presentation here.

The Managed Services Engine is a CTP technology that acts as a service intermediary. It helps abstract consumers form the services that they call, performing things such as xslt mapping, protocol switching, version mapping and service stubbing (useful for testing unhappy service respone paths as well as development of the consumerbefore the service is really ready.)

You can get the Managed Services Engine on CodePlex: http://www.codeplex.com/servicesengine

MockingBird- Beta 2

MockingBird- Beta 2

MockingBird – Beta 2 is now available.
The changes/ fixes are as follows
(1) The system can handle remote WSDL’s now (using DiscoveryClientProtocol). [But this will use the Default network credentials to get the WSDL. There are no facilities as yet for providing this information as settings for the tool).
(2) Fixed the WSDL Browser which crashed when […]

C# for Stream to byte[]

I found this method, I’ve used it a few times, so I’m going to put it here for my own use.
/// <summary>
/// Reads data from a stream until the end is reached. The
/// data is returned as a byte array. An IOException is
/// thrown if any of the underlying IO calls fail.
/// </summary>
/// <param name="stream">The stream to read data from</param>
/// <param name="initialLength">The initial buffer length</param>
public static byte[] ReadFully (Stream stream, int initialLength)
{
    // If we've been passed an unhelpful initial length, just
    // use 32K.
    if (initialLength < 1)
    {
        initialLength = 32768;
    }
    
    byte[] buffer = new byte[initialLength];
    int read=0;
    
    int chunk;
    while ( (chunk = stream.Read(buffer, read, buffer.Length-read)) > 0)
    {
        read += chunk;
        
        // If we've reached the end of our buffer, check to see if there's
        // any more information
        if (read == buffer.Length)
        {
            int nextByte = stream.ReadByte();
            
            // End of stream? If so, we're done
            if (nextByte==-1)
            {
                return buffer;
            }
            
            // Nope. Resize the buffer, put in the byte we've just
            // read, and continue
            byte[] newBuffer = new byte[buffer.Length*2];
            Array.Copy(buffer, newBuffer, buffer.Length);
            newBuffer[read]=(byte)nextByte;
            buffer = newBuffer;
            read++;
        }
    }
    // Buffer is now too big. Shrink it.
    byte[] ret = new byte[read];
    Array.Copy(buffer, ret, read);
    return ret;
}

SBUG Meeting next Tuesday

Just a quick reminder for the SBUG meeting next Tuesday. We will be joined by:

Tony Meleg from the Dublin Team doing an overview of Dublin following on from the TechEd Dublin sessions

John Plummer from Microsoft UK hosting a Q&A session

Remember this is an in person meeting at Cardinal Place, but for those who cant make it in person we will also have live meeting available so you can still log in and join us.

The full event details are on the following page:

http://sbug20090526.eventbrite.com/

BizTalk Adapter Pack v2.0 Released

BizTalk Adapter Pack v2.0 Released

The final release of the BizTalk Adapter Pack version 2.0 is now available on MSDN and TechNet for subscribers. The single download contains both 32bit and 64bit versions.
Overview:
The Microsoft BizTalk Adapter Pack contains adapters that enable enterprise applications and databases to interface with each other by implementing a common adapter framework. They expose line of […]

Biztalk 24×7 – new version goes live

Saravana Kumar has just re-launched BizTalk 247.com (see http://www.biztalk247.com) with lots of new content and a new layout, look and feel. Look very quickly and you might just spot that awful photo of me on the home page! The one with the bright purple tie and a dark blue shirt. What was I thinking? Oh well, no-one said that being a BizTalk user equated to being cool. And I desperately needed a haircut.
Anyway, congratulations to Saravana. This is a really useful single stop shop for BizTalk resources.Lots of videos (including a couple with me waving my hands around like a manic thing) and links to various widgets, featured books and guides and tons of other stuff. Saravana has also worked hard to ensure that URLs to various resources on the site are short and to the point so that you easily find what you are looking for. Mind you, being a point & click man, myself, I think the site is very easy to navigate anyway.

BAM Archiving/BAM_DM_<ActivityName> observations

From the BizTalk documentation: “The DTS package, BAM_DM_<ActivityName>, performs the partitioning and archiving/purging. Each time this package runs, it truncates another partition and archives/drops all partitions that are outside the online window.”

Here’s my restatement of this: The BAM_DM_<ActivityName> package (SSIS or DTS depending on your version of BizTalk and despite the fact that the docs refer to DTS only) performs two operations: building partition tables and archiving those partitions to the BAMArchive database when a partition’s creation date is past the online window.

If you open up a BAM_DM_<ActivityName> SSIS/DTS package and walk through it you’ll see that the second step calls the bam_Metadata_SpawnPartition stored procedure. Toward the end of that sproc it inserts a new record into the bam_Metadata_Partitions table setting the CreationTime’s value but not the ArchivedTime value which means it will be NULL.

A later step in the package calls the bam_Metadata_BeginArchiving stored procedure. Toward the end of this stored procedure it performs an SQL Update on the bam_Metadata_Partitions table where the ArchivedTime is null and the CreationTime is less than the online threshold (derived from the online window.) If the Update succeeds any partitions that should be archived have their ArchivingInProgress value set to 1 (this is set so that if the package runs again, before it completes, the second run will essentially just exit without doing anything.)

Later steps (two of them) in the package perform the actual archiving; they examine the CreationTime and ArchivedTime to figure out what they should archive, using the same logic outlined above (for the . bam_Metadata_BeginArchiving sproc).

There are many more details but hopefully provides enough detail for the takeway.

Takeaway: your BAM_DM_<ActivityName> package almost certainly needs to run twice to actually archive data to the BAMArchive database because the time between when the partition is created during the second step of the package and when archiving will happen (later steps) isn’t going to be greater than your online window. The lowest you can set your online windows is 1 minute. Unless you have massive quantities of BAM data, and a slow SQL Server, it isn’t going to take more than a minute between when the bam_Metadata_SpawnPartition sproc runs (which sets the CreationTime) and the archiving steps execute. So, the first time you run your package it will create the partitions; the second time it will archive those partitions, if they fall outside the online window. This means you need to think carefully about how often you schedule your BAM_DM_<ActivityName> packages and how that is related to the online window.

For additional context reference the Business Activity Monitoring in Depth for Developers whitepaper here: http://download.microsoft.com/download/D/D/A/DDA95D1F-14D4-49A2-B2C8-E244535E8502/BAM_for_Developers.docx

Interesting BizTalk links

Interesting BizTalk links

Details for Version 2.0 of the BizTalk ESB Toolkit (Brian Loesgen)

Complex-Event Processing (CEP) Explained for BizTalk Users (Charles Young)

Interview Series: Four Questions With Ewan Fairweather (Richard Seroter)

BizTalk Server Best Practices Analyser v1.2 (Released 5/6/2009)

BizTalk Hotrod (Issue 6, April 2009)

Index:

  • Writing Great BizTalk Applications
  • BizTalk Rule Engine, a practical application
  • Development Challenges with XML over AS2
  • Batching Outbound Messages
  • Hierarchical naming convention
  • To Be, Or Logical Not To Be
  • Enhancing the BizTalk Mapper
  • Muenchian Grouping and Sorting in XSLT
  • Eliminate BizTalk Admin Problems with Terminator
  • Static Code Analysis for BizTalk Using BizTalkCop
Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

Microsoft Presentation on CEP and BizTalk

After publishing the article on CEP on the Microsoft platform a couple of days ago ( see http://geekswithblogs.net/cyoung/archive/2009/05/18/cep_explained_for_biztalk_users.aspx), I was reminded of an excellent presentation given by my good friends John Plummer and Jeff Johnsonat the Microsoft Architect Insight conference in 2008. John and Jeff both work for Microsoft in the UK. They used NEsper (the .NET port of the open source Esper event processing engine) to demonstrate how CEP might be used in conjunction with BizTalk technologies. I dug out the slides and reminded myself of their presentation. All very good stuff. the slides are at:

http://download.microsoft.com/documents/uk/msdn/events/SOL/SOL08.pptx