Oslo based solution for deploying BizTalk applications

by | Mar 3, 2009 | BizTalk Community Blogs via Syndication

Since PDC I’ve been working on and off on an “Oslo” based solution for deploying a BizTalk application; unfortunately I couldn’t get a good chunk of time to play with this, so it’s been dragging a bit, but I’m getting close, so here are some details –

I’m a big advocate of automated builds; it’s a topic that probably deserves a post of its own, so I won’t get started on this here, but the idea is that one must have a way to be confident that, when its time to [re-]deploy the app, it will get deployed successfully time after time, without a hitch;

From my experience, deploying a BizTalk application of a decent size is often not trivial, and often it does not go smoothly if done manually (things are being missed, done out of order, wrong versions being picked up, etc), which does not help to boost the confidence in the solution (or BizTalk as a whole) in the organisation; automating the process can save a lot of time (can be done unattended, while in a meeting, out for lunch or overnight), save a lot of head scratching, boost the confidence in the solution and set the ground for proper automated builds, auto testing etc.

To that extent, I have previously built an MSBuild based framework for deploying BizTalk applications.

While I’m sure it does not suit all purposes as it was developed to support those scenarios I had, I think it’s quite comprehensive and had served me well over the past couple of years.

It allows one to provide an MSBuild-formatted file with some parameters and lists and, using the pre-created framework, would do things like remove the existing application (after terminating current instances and un-enlisting services), build the new solution(s), deploy assemblies to BizTalk, add/import binding files, start ports etc.

This is working great for us, and we’ve been using it extensively for quite a while now, but there’s one major downside – it requires one to maintain those MSBuild scripts.

Now – MSBuild is cool, I do like it very much, but as it’s generic – it does not speak in BizTalk terms, and as it’s XML – it’s quite verbose, and I wanted an easier format for all of us to work with.

So – inspired after seeing Oslo and even more driven after visiting PDC – I’ve decided to come up with an MGrammar based language for describing a BizTalk application to be deployed.

So far I’ve come up with “version 1.0” of my grammar that allows to describe the major artifacts in a BizTalk application and a basic runtime to process source files written in this language, the language allows you to describe things like –

  • References to other applications
  • Solutions and projects to build
  • BizTalk assemblies to deploy
  • Orchestrations contained in those (these can then be terminated before attempting to remove the application prior to deployment)
  • .net assemblies to deploy
  • Binding file to import
  • Binding files to add (and the name of the environment to attach to those)

As Oslo hasn’t RTM-ed yet, I can’t quite rely on it yet, and so I cannot use it in production in any shape or form, or can I?
I found a good middle ground for us which allows us to gain from the benefits I’m hoping to get by using my language, while not exposing ourselves too much to the risks of using early technology –

At the moment the runtime I’ve built for this is used to generate the MSBuild scripts we’re already using out of the source files; in this way – if Oslo disappears tomorrow, or significantly changes (not that I think that’s going to be the case), we’re safe – we still have the MSBuild scripts as “checked-in artifacts” and so we have lost nothing.

So – how does an instance of my language looks like? here’s an example: 

Application MyApp
{
    //add references to other applications
    add reference SomeOtherApp;
    add reference AndAnother,andAThird;
    
    //build the solution
    build "SomeFolder\SomeOtherFolder\SomeSolution.sln";
    build "SomeFolder\SomeOtherFolder\SomeProject.btproj";
    
    //add required biztalk assemblies
    add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Schemas.dll"
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;

    add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Maps.dll"
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;

    add biztalk assembly "SomeFolder\SomeOtherFolder\bin\deployment\Orchestrations.dll" 
    with orchestrations
        {
            MyApp.SomeOrchestration,
            MyApp.AnottherOrchestration,
            MyApp.AndAnotherOrchestration
        }        
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;
    
    //and .net helpers assembly
    add assembly "SomeFolder\SomeOtherFolder\bin\release\HRG.FareWatch.Hotel.Helpers.dll" 
        version=1.0.0.0 
        culture=neutral 
        publicKeyToken=314bd7037656ea65;
    //import dev bindings
    import binding "SomeFolder\SomeOtherFolder\Bindings.xml";

    //add various environment's bindings.
    add binding "SomeFolder\SomeOtherFolder\Bindings.Dev.xml" "development";
    add binding "SomeFolder\SomeOtherFolder\Bindings.Stage.xml" "stage";
    add binding "SomeFolder\SomeOtherFolder\Bindings.Prod.xml" "production";
}

as I’ve said – I already have a runtime that generates the MSBuild scripts required to deploy these – the runtime outputs several files to the temp folder –

  • The “framework” MSBuild script that contains all the targets I’m using
  • The Microsoft.Sdc.Tasks.BizTalk.dll and the Microsoft.Sdc.Tasks.dll which contains many useful custom tasks.
  • The relevant import file for the SDC tasks
  • The generated MSBuild file that contains all the various artifacts based on the source code

That latter file, which is the equivalent of my source code, and what we had to maintain so far, looks like – 

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="DeployAll">
  <Import Project="Deploy.Framework" />
  <ItemGroup>
    <MyApp Include="SomeOtherApp" />
    <MyApp Include="AndAnother" />
    <MyApp Include="andAThird" />
  </ItemGroup>
  <ItemGroup>
    <Bindings Include="General">
      <path>SomeFolder\SomeOtherFolder\Bindings.xml</path>
    </Bindings>
  </ItemGroup>
  <ItemGroup>
    <EnvironmentBindings Include="General">
      <path>"SomeFolder\SomeOtherFolder\Bindings.Dev.xml"</path>
      <environment>development</environment>
    </EnvironmentBindings>
    <EnvironmentBindings Include="General">
      <path>"SomeFolder\SomeOtherFolder\Bindings.Stage.xml"</path>
      <environment>stage</environment>
    </EnvironmentBindings>
    <EnvironmentBindings Include="General">
      <path>"SomeFolder\SomeOtherFolder\Bindings.Prod.xml"</path>
      <environment>production</environment>
    </EnvironmentBindings>
  </ItemGroup>
  <ItemGroup>
    <ExternNetAssembly Include="HRG.FareWatch.Hotel.Helpers">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\release</path>
    </ExternNetAssembly>
  </ItemGroup>
  <ItemGroup>
    <Solution Include="SomeSolution">
      <path>SomeFolder\SomeOtherFolder</path>
    </Solution>
    <Solution Include="SomeProject">
      <path>SomeFolder\SomeOtherFolder</path>
    </Solution>
  </ItemGroup>
  <ItemGroup>
    <BTSProject Include="Schemas">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\deployment</path>
    </BTSProject>
    <BTSProject Include="Maps">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\deployment</path>
    </BTSProject>
    <BTSProject Include="Orchestrations">
      <Version>1.0.0.0</Version>
      <Culture>neutral</Culture>
      <PublicKeyToken>314bd7037656ea65</PublicKeyToken>
      <ProcessorArchitecture>MSIL</ProcessorArchitecture>
      <path>SomeFolder\SomeOtherFolder\bin\deployment</path>
    </BTSProject>
  </ItemGroup>
  <PropertyGroup>
    <BTApplicationName>MyApp</BTApplicationName>
    <!-- Set for a remote deployment -->
    <!-- Deploying BizTalk Server name - leave blank if local-->
    <!-- not currently supported by runtime-->
    <BTServerName>
    </BTServerName>
    <!-- Deploying BizTalk Server database - leave blank if BizTalkMsgBoxDb-->
    <BTServerDatabase>
    </BTServerDatabase>
    <!-- Deploying BizTalk Server SQL user name - leave blank if local-->
    <BTServerUserName>
    </BTServerUserName>
    <!-- Deploying BizTalk Server SQL password - leave blank if local-->
    <BTServerPassword>
    </BTServerPassword>
  </PropertyGroup>
</Project>

Would you agree that the former source code is easier to maintain?

note: one thing you would notice is that none of the paths contains a root; I assume that this would be used by different developers/IT pros which may have the code in a different path; however, as the assumption is that the code will come from source control, my framework script expects you to provide the root path to your source code and assumes the specified paths start there.


So – what’s next?

From a coding perspective – I plan to add support in my runtime to perform the actual deployment; this is I would like it to do once Oslo is in production so I’ll add it as an option through a command line switch.

This option would tell the runtime to deploy the artifacts to a specified BizTalk server using BTSTask or the Object model instead of generated the MSBuild script.

I also want to make some modification to my language definition to make the MGraph produced cleaner (and better geared towards using the repository and Quadrant at a later stage) as well as, obviously, add support for more features, after which I plan to publish both my language and the runtime somewhere (here, codeplex, DevCente..we’ll see)

I’ll post some more notes on all of these here in the near future hopefully

The Twitter Search API made easy with Linq to XML

by | Mar 2, 2009 | BizTalk Community Blogs via Syndication

My pal Lieven and I are preparing some cool demos to show at the Belgian TechDays SharePoint Preconference next week. Our goal is to build up a nice story that shows the various ways how to use SharePoint 2007 in combination with .NET 3.5. For some those demos we’re going to make use of one of today’s hypes: Twitter! This evening I tried to query Twitter data using Linq and I discovered there are lots of samples available on the net; there’s even a complete Linq to Twitter provider on CodePlex. Too bad the Linq to Twitter provider is focused on the tweets for a specific Twitter user and I wanted to make use of Twitter’s Search API. It turned out to be very, very easy (thanks to Robert Horvick’s code snippets). Just dump the code at the bottom of this post in your .NET 3.5 project, and you can query the Twitter Search API using ‘sharepoint’ as the query like this:


foreach (U2U.Demos.TwitterSearch.Tweet t
    in U2U.Demos.TwitterSearch.Query(“sharepoint”))
{
    MessageBox.Show(t.Title);
}


Here’s the code for the TwitterSearch class:


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml.Linq;


namespace U2U.Demos
{
    public class TwitterSearch
    {
        private const string urlTemplate = “http://search.twitter.com/search.atom?q={0}”;
        private static XNamespace atomNS = “http://www.w3.org/2005/Atom“;


        public static List<Tweet> Query(string query)
        {
            XDocument xDoc = XDocument.Load(string.Format(urlTemplate, query));


            var tweets =
            (from tweet in xDoc.Descendants(atomNS + “entry”)
              select new Tweet
              {
                  Title = (string)tweet.Element(atomNS + “title”),
                  Published =
                    DateTime.Parse((string)tweet.Element(atomNS + “published”)),
                  Id = (string)tweet.Element(atomNS + “id”),
                  Link = tweet.Elements(atomNS + “link”)
                      .Where(link => (string)link.Attribute(“rel”) == “alternate”)
                      .Select(link => (string)link.Attribute(“href”))
                      .First(),
                  Author = (from author in tweet.Descendants(atomNS + “author”)
                            select new Author
                            {
                                Name = (string)author.Element(atomNS + “name”),
                                Uri = (string)author.Element(atomNS + “uri”),
                            }).First(),
              });


            return tweets.ToList<Tweet>();
        }


        public class Tweet
        {
            public string Id { get; set; }
            public DateTime Published { get; set; }
            public string Link { get; set; }
            public string Title { get; set; }
            public Author Author { get; set; }
        }
        public class Author
        {
            public string Name { get; set; }
            public string Uri { get; set; }
        }
    }
}

Using the WCF SQL Adapter in .NET: Calling Stored Procedures

by | Mar 2, 2009 | BizTalk Community Blogs via Syndication

[Source: http://geekswithblogs.net/EltonStoneman]

The Adapter Pack 2.0 for BizTalk has been released in public beta recently, and among the WCF Line Of Business adapters it contains the WCF SQL adapter. This exposes SQL Server connections as WCF service endpoints, and lets you connect to a SQL Server source using the standard ServiceModel stack. The adapter pack will be released under the BizTalk brand, but the adapters themselves are not limited to BizTalk – the WCF SQL adapter can be used natively in .NET code.

This is a brief walkthrough covering the SQL Adapter for executing stored procedures, and I’ll be covering SQL statements in a subsequent post.

Installation

Installation of the Adapter Pack is straightforward – you’ll need .NET 3.5 Service Pack 1 with the latest hotfixes applied, and you’ll need to install the WCF LOB Adapter SDK and then the BizTalk Adapter Pack 2.0 Evaluation (the beta version is limited to 120-day use). Note, you do not need to have BizTalk installed, and the tooling to support the WCF LOB adapters runs under Visual Studio 2008 as well as 2005. Help files for all the adapters are included, and although in pre-release form they are detailed and thorough.

Walkthrough: Consuming a Stored Procedure

The Adapter Pack adds a new context menu to code projects in Visual Studio – Add Adapter Service Reference. Run this and you’re given a generic form for configuring your WCF LOB adapter. Choose sqlBinding to set up a WCF SQL connection:

Click Configure and you specify the connection configuration that will be used to build the binding. In the case of the WCF SQL adapter, you need to specify:

  • Client Credential Type (Windows for integrated authentication);
  • Server (database server name);
  • Instance (SQL instance name, if configured);
  • Initial catalog (the database to connect to).

This will build you a URI of the form: mssql://<server>/<instance>/<initialCatalog>? – with the ending question mark used to separate the core connection details from any configuration options.

Click Connect and you have the option to generate a Client binding, for making outbound requests to SQL Server (executing SQL statements, stored procedures etc.), or a Service binding which will react to inbound calls from SQL Server (for Query Notification or polling). Choose Client and the category view will be populated with a hierarchy of database objects which can be generated as WCF client proxies:

Generated Code

For stored procedure calls, the adapter can create generic proxies for weakly-typed calls returning populated DataSets, or strongly-typed calls which will generate entities representing the return from the call. In this case I’ve selected a Strongly-Typed Stored Procedure called GetManufacturer; add the selection and with the default options the adapter generates two items:

  • App.config – containing the WCF binding configuration;
  • SqlAdapterBindingClient.cs – containing the generated entity types and proxy classes.

The full binding configuration for the SQL adapter looks like this:

<system.serviceModel>

<bindings>

<sqlBinding>

<binding name=“SqlAdapterBinding“ closeTimeout=“00:01:00“ openTimeout=“00:01:00”

receiveTimeout=“00:10:00“ sendTimeout=“00:01:00“ maxConnectionPoolSize=“100”

encrypt=“false“ workstationId=“” useAmbientTransaction=“true”

batchSize=“20“ polledDataAvailableStatement=“” pollingStatement=“”

pollingIntervalInSeconds=“30“ pollWhileDataFound=“false“ notificationStatement=“”

notifyOnListenerStart=“true“ enableBizTalkCompatibilityMode=“true”

chunkSize=“4194304“ inboundOperationType=“Polling“ useDatabaseNameInXsdNamespace=“false”

allowIdentityInsert=“false“ enablePerformanceCounters=“false”

xmlStoredProcedureRootNodeName=“” xmlStoredProcedureRootNodeNamespace=“” />

</sqlBinding>

</bindings>

<client>

<endpoint address=“mssql://x/y/z?“ binding=“sqlBinding”

bindingConfiguration=“SqlAdapterBinding“ contract=“TypedProcedures_dbo”

name=“SqlAdapterBinding_TypedProcedures_dbo“ />

</client>

</system.serviceModel>

– note that the binding contains some familiar WCF settings (sendTimeout, receiveTimeout), but the majority are SQL Server specific connection options. The client element specifies the contract as TypedProcedures_dbo, the ServiceContract interface generated by the adapter, which has a single OperationContract defined:

GetManufacturerResponse GetManufacturer(GetManufacturerRequest request);

The proxy code for the client is all generated, so to invoke the stored procedure in your own code it’s a familiar case of instantiating the client and calling the service, and of course you have full IntelliSense on the entity representing the resultset:

It’s the content of the generated code that’s interesting. The service, request, response and entity objects here are contained in 166 lines of generated code. The entity object is just a plain DTO-style class which implements IExtensibleDataObject to allow access to any returned data that hasn’t been mapped, and has a DataContract attribute with the schema name representing the stored procedure. The mapping between the entity properties and the returned columns is done with standard System.Runtime.Serialization attributes, so the ManufacturerId column is represented as:

[System.Runtime.Serialization.DataMemberAttribute()]

public System.Nullable<short> ManufacturerId {

get {

return this.ManufacturerIdField;

}

set {

this.ManufacturerIdField = value;

}

}

– note that this is an optional field in the database table, so it’s generated as nullable in the entity. No other flags or code are used to map data, so the WCF SQL adapter is effectively deserializing the resultset from the stored procedure call straight into the DataContract.

The generated code works well and is cleanly produced, but it has a few quirks you may not be happy with. A typed client class is generated for each individual procedure, whereas you might want them grouped into a single class which represents the full suite; and the class names are a bit cumbersome (“StoredProcedureResultSet0”, “TypedProcedures_dboClient”). However, the code needed to actually connect to SQL through WCF and map the response is so simple that it’s a straightforward task to generate your own code from custom templates.

Potential Usage

After an initial look, the WCF SQL adapter seems to be an attractive option for generating and powering the data access layer of a .NET application, entirely apart from its primary purpose as a BizTalk adapter. It repositions data access as a service call and uses the standard WCF mechanisms of ServiceContract and DataContract for information exchange. Assuming other data providers follow suit, or other WCF-database adapters follow from the community, it’s a nice way of isolating your application from the physical database, so swapping to MySQL or Oracle could become a simple matter of changing your WCF binding.

It’ll be interesting to see the licensing of the WCF LOB adapters from Microsoft. Currently the availability of a comprehensive suite of adapters is being positioned as one of the attractions of BizTalk as the Integration Server, compared to WF+WCF+Dublin as the Application Server. With the WCF SQL adapter there’s a lot of potential take-up as a simpler alternative to the ADO.NET Entity Framework, so if it requires a BizTalk license, there will be room for an open source alternative.

nHibernate Tutorials for .Net!

nHibernate Tutorials for .Net!

by | Mar 2, 2009 | BizTalk Community Blogs via Syndication

Often these days, i find managers requesting developers to get their hands on Hibernate methodologies to connect to various relational databases. As i was trying to check this out, i came across this very useful tutorial on nHibernate
@ http://www.hibernate.org/362.html
NHibernate handles persisting your .NET objects to and from an underlying relational database. Rather than you […]

SQL Server 2005/2008 Best Practices!

SQL Server 2005/2008 Best Practices!

by | Mar 2, 2009 | BizTalk Community Blogs via Syndication

Hi All,
I found a list ofBest Practices available for SQL Server 2005/2008. Check them out here:
XML Best Practices for Microsoft SQL Server 2005%u00b7 Performance Optimizations for the XML Data Type in SQL Server 2005%u00b7 SQL Server 2005 Security Best Practices – Operational and Administrative Tasks%u00b7 SQL Server 2005 Full-Text Queries on Large Catalogs: Lessons […]

Xml Schema Documenter

Xml Schema Documenter

by | Mar 2, 2009 | BizTalk Community Blogs via Syndication

I found Sand Castle to be a really useful tool for documenting our projects and now i am more excited to see this new tool called Xml Schema Documenter (a plugin of sandcastle) which documents xml schemas.

XML Schema Documenter allows you to easily create XML schema documentation by extending Sandcastle Help File Builder.
XML Schema Documenter […]