Initial customer feedback on BizTalk Server 2009

As most of you may know from a previous entry in this blog, the first beta release of BizTalk Server 2009 has been made publicly available.


You can download it directly, after signing with a Live ID, from here.


We also made available the first CTP of the BizTalk 2009 ESB Guidance in codeplex.



A major theme of this release is the enhanced developer productivity and the significantly improved developer team experience.


This was made possible through integration with Microsoft’s application life-cycle management (ALM) solution: Visual Studio Team System and Team Foundation Server.


I wanted to share with you the exciting feedback from an early adopter.
Gulf Coast Seal Ltd. took part in the Technology Adaption Program (TAP) of BizTalk Server 2009. Today their solution is already in production.
Here is what Jeff Lynch, Gulf Coast Seal’s Information Technology Manager, has to say about BizTalk Server 2009:




 


“I really can’t tell you just how happy using BizTalk Server 2009 as a fully supported part of VSTS and TFS makes me.
Having BizTalk Server 2009 seamlessly fit into our application development life cycle under Team Foundation Server is extremely important in terms of our confidence in using BizTalk as the “glue” which binds our business processes together. Today, over 48% of our sales transactions and over 25% of our entire revenue stream comes in via BizTalk Server and to say that it’s “vital to our business” is an understatement.”


 


“In case I haven’t already mentioned this. The build and deploy speed of this beta is excellent. I just built and deployed a four project solution in less than 30 seconds in a virtual machine. This is substantially faster than BizTalk Server 2006 R2. Even my complex solutions are building very quickly and deploying in a matter of minutes. It is also nice to see the deployment detail being written to the Output window in Visual Studio 2008.”




 


I thought this might encourage you to start downloading and testing the Beta version of BizTalk 2009 if you have not done so already J


 


Ofer

Developing Integration Solutions with BizTalk Server 2006 R2

Here is some good news for veteran BizTalk developers & architects.
Remember the whitepaper: Developing Integration Solutions with BizTalk Server 2004 ?


I used to recommend it to any organization embarking on their first BizTalk project.


It contained an approach for the major part of the Application Life Cycle Management for BizTalk Server 2004.


Well, recently Dennis Mulder, a senior consultant with MCS has updated this document to BizTalk 2006 R2 titled:
Developing Integration Solutions using BizTalk Server 2006 and Team Foundation Server
Check out the newly updated document that provides
techniques and best practices for designing, developing, and deploying solutions within Microsoft%u00ae BizTalk%u00ae Server 2006 (R2) in conjunction with Team Foundation Server.


We intend to create next year yet another version of this document to cover BizTalk Server 2009.


 


Enjoy it!


 


Ofer

Ring in the New Year with the SOA & Business Process Conference – goes great with Eggnog

We had our first real snow last week which means that ski season is finally upon us and everyone in Seattle must relearn how to drive in slightly awkward conditions (again). This is a great time of year as work tends to slow a bit, winter sports heat up and you can still log plenty of miles on the bike as long as you pay attention to the drivers on the bottom of the learning curve.  While 2008 is a year that few are likely to forget, most people are already getting ready for 2009. With the new year coming there is one more thing I am excited about – and that’s the upcoming SOA and Business Process Conference on January 28-29, 2009. 


 


Given the current economic environment, we know that many of our partners and customers are cutting back on travel. Thus, we are trying out a completely different format -we are still holding an event in Redmond (dates mentioned above) though the format will be a new, concentrated, two-day version that will provide our customers and partners with the most relevant information possible. We’re also taking the conference on the road.  Some of you have already attended events scattered around the globe.   Why are we doing this?  First and foremost, a geographically distributed schedule allows us to interact with far more people.  Second, making the event itself more specific will maximize time for those who make the investment to come.


 


During the conference, attendees will see and hear how customers and partners are using real-world SOA to drive business results, and will also have the opportunity to hear from customers and partners who’ve successfully implemented real-world SOA solutions, Microsoft execs and Gartner analyst Nick Gall on maximizing current SOA investments. For those of you who can’t make it out, we’re planning  to provide new SOA and BPM content via the web toward the end of February.


 


Feel free to visit our site to see the full session list and other information. Hope to see you all there.

Charles Young video released on Channel9

Charles Young came to hold two sessions at the Swedish BizTalk User Group in December this year. The topic came out to be about Orchestration Patterns in general, and was very appreciated by everyone in the audience. Charles Young is an MVP and software developer specializing in .NET technologies. He works for SolidSoft Ltd. in the UK, and is the author of one of the most appreciated BizTalk blogs on the web.

We are now happy to share these sessions with everyone who couldn't make it (along with the almost 90 people who did make it). This is the part one of two. The second session is coming up soon.

http://channel9.msdn.com/posts/MSDNSweden/10-mest-anvndbara-orchestration-patterns-del-1-av-2/

As we've come to the end of the year, It's amazing to think back of all the great international speakers we've had coming to our user group. Darren Jefford, Dwight Goins, Jon Flanders and now Charles Young… It's going to be a challenge to top that next year.

BizTalk 2009 thoughts…

Have you had a chance to play
yet
?

Many folks have noted that a lot of the Visual Studio elements that have been present
within BizTalk to support the development experience are no longer…quite so BizTalk
specific!  BizTalk projects now build upon C# projects, and thus a lot of the
differences that you used to see in navigating property pages, compilation settings
and build mechanics are now gone.  This is a very good thing – it allows you
to leverage skills you already have on your team.

MSBuild support is now first class.  Everyone who went through the trouble to
install (and invoke) DevEnv.exe on their build server in order to build BizTalk projects
will be glad to know that this is no longer required.  (Builds can be ever so
much faster when you aren’t relying DevEnv…)

Because of the close relationship with C# projects, you can now have C# artifacts
directly in your BizTalk projects.  Many people have noted that when you “Add
new item…”, C# classes aren’t offered as an option.  The product group has
explained that this is because currently, the designers (such as the Orchestration
designer) are unable to provide intellisense for types that are within the same assembly. 
So, you are required to use “Add existing item…”.

(This sort of reminds me of the interaction between
pipelines and schemas – the former require fullly qualified assembly information at
run time, which they won’t get if you combine these two artifacts in the same assembly.)

If this limitation persists, will the feature get used?  I tend to think so. 
There are often cases where the smart thing to do in an orchestration is to delegate
to a component…but the work you do in that component is, at times, so specific to
the orchestration that it makes sense for them to be co-located for deployment and
organizational purposes.  What do you think ?

The support for unit testing is extremely welcome – check it out.  Debugging
maps is great as well, but I’m now (often) partial to the external
XSLT
approach these days.

BizTalk Adapter Pack v2 CTP

BizTalk Adapter Pack team has released a CTP. This is  the 5th CTP and the first one available publicly to everyone to try.  The specific enhancements in this CTP  are mostly to the OracleDB adapter(included in the BAPv1) to which we added features like support for polling stored procedures and composite operations. In addition we have addressed some feedback we got from the other adapters .


 


To re-iterate here is the complete list of features BAPv2 is adding . The downloads are available internally at http://adapters and on the connect site


 


BizTalk Adapter Pack V2 – Themes


 


New Adapters


%u00b7           Oracle EBS Adapter


%u00d8  Browse and search metadata for each Oracle application


%u00d8  Support Interface tables, Interface views


%u00d8  Support for PL/SQL APIs


%u00d8  Concurrent Programs, Request Sets


%u00d8  Support for polling tables, Stored Procedures


%u00d8  Notification support


%u00d8  App context initialization


%u00d8  Supports REF cursors, User Defined Types, LOB types, RECORD types


%u00d8  Support for PL/SQL Tables, boolean data types


%u00d8  Support for MLS EBS installations


%u00d8  Microsoft Office SharePoint Server Integration


%u00d8  Versions supported – ODP.NET  11.1.0.6.21 (available only on 32bit), Oracle EBS 11.5.9, 11.5.10 and 12


%u00b7           SQL Server Adapter


%u00d8  Insert/Update/Delete/Select operations on tables and views


%u00d8  Invoke (SQL and CLR) Stored procedures, scalar valued functions , table value functions


%u00d8  Supports execution of generic T-SQL statements, via the ExecuteReader(), ExecuteNonQuery() and ExecuteScalar() operations


%u00d8  Configurable polling – polling statement, polling interval, receive multiple result sets


%u00d8  Support for x86 and x64  platforms


%u00d8  Support for new data types in SQL2005 and SQL2008 (notably FILESTREAM, varbinary (max))


%u00d8  Support for UDTs


%u00d8  Ability to pass table value parameters


%u00d8  Composite operations – invoke operations on multiple tables and call any number of stored procedures in the same transaction


%u00d8  SQL query notifications


%u00d8  Ability to directly call stored procedures which used the ’for xml’ style supported by the BizTalk 2006 SQL adapter to ease backward compatibility


%u00d8  Versions supported – SQL 2000, SQL 2005, SQL 2008


 


Enhancements to the existing V1 Adapter Pack


 



  • Include hot fixes built since the V1 Adapter Pack released

  • Support new Microsoft Platform releases – BTS 2009, VS 2008, Windows Server 2008

  • Oracle DB Adapter:

%u00d8  Polling support for tables, Stored procedures


%u00d8  UDTs, PL/SQL tables  (from Beta onwards)



  • SAP Adapter:

%u00d8  Support for Table Types


%u00d8  Customization of adapter behavior when “special” values obtained for DATS/TIMS/NUMC


%u00d8  SAP ADO Provider Enhancements:


o   Support for SSRS


o   Support for SAP Queries


o   Ability to specify the result set to give to SSIS when executing BAPIs



  • Siebel Adapter:

%u00d8  MVG operations (associate, dissociate, query) in design time enabled


%u00d8  Bounded pick list fields supported per recommended approach


%u00d8  Context specific custom messages generated for exceptions


 

First Look at BizTalk 2009 Build and Developer Experience

We had a very useful brown bag session with the BizTalk Product Team this week and went through some of the new developer improvements. Fortunately not much of this was under NDA so I thought id share some of what I learnt and my thoughts on this. Ill just keep this in the simple format below:

BizTalk Project Files are now MsBuild

A new BizTalk project file is basically an extension of a C# project file. It looks very similar to a normal MsBuild based project file see the Picture below.

From this picture you can see lots of familiar stuff. You can see from the following picture from the project file that a BizTalk project file imports the C# MsBuild targets as well as some new BizTalk ones by default.

I believe BizTalk projects have their own project type guid in the project file system which indicates them as a certain type of project file. You can see this above in the MsBuild script where the Project type Guids indicate this project contains C# and BizTalk stuff. I would assume here you can also add the project type guid for MsTest projects and then start putting tests in this project also. In practice you probably wouldn’t do this, but it illustrated whats possible with project files now being in MsBuild.

BizTalk Projects Can have non BizTalk Artefacts in them

That’s right you can now add other artefacts than BizTalk ones. Some examples of this might be you have a C# class, a text or xml file in your project. Because of what I said above about the C# targets this means if you include a .cs file in your BizTalk project then it will be compiled and made available just like any other normal class. There are a couple of questions and limitations relating to this however:

  1. In the current CTP you can not Add New Item and add a C# file etc to the project. This is expected to be in future releases, but for now you will need to add the item to the file system then add it as an existing item.
  2. Because the BizTalk project file extends on C# if you add a VB file it will not by default be compiled. If however you add the VB MsBuild targets it should be possible to compile these files also. Essentially its just an MsBuild script so you can do what ever you can do with .net projects.

Assembly Info

BizTalk projects now have AssemblyInfo.cs type files like any normal C# project. This makes things simpler for manipulating these kind of files. See the pic below, if you notice there is a new assembly attribute called BizTalkAssemblyAttribute which I guess indicates the assembly is a BizTalk assembly.

No more Deployment/Development

One thing which makes me very happy is that the pointless configuration difference between C# and BizTalk projects is now gone. You can see this below.

It would be nice if the Deploy tick box wasn’t checked by default also. This is a useful setting for small demo type projects but in real projects you are unlikely to deploy your BizTalk assemblies through the Visual Studio IDE. I guess now we are based on MsBuild it might be possible to create a new project template where this is not checked (possibly a small community project here).

Compile BizTalk Solutions without Visual Studio

In the installation of BizTalk there is an option to install Project Build components. This means you can use these components to build your solution without Visual Studio. This is what a number of people have been asking for to help them implement the good practice on a Build server of compiling code without needing Visual Studio.

Strong Naming Change

The way BizTalk projects are strong named has changed and is now the same as for C# projects. From the picture below you can see this is set through the project property editor and it sets information in the MsBuild project file. Again another handy improvement in that it is just the same as C#.

Common Project Settings

As you can see from the below picture you can see the BizTalk project also has the same settings which are commonly used in C# projects. You can set the highlighted sections so you can set warnings and constants etc.

One of the more interesting things however though is that you can setup the XML documentation for the project. At present this will only do documentation for things like C# files within the project, but I guess in the future this is something that either Microsoft or the community may look at as a way of incorporating BizTalk Documenter style functionality into the compilation process.

Project Migration

When you migrate your project to BizTalk 2009 there is a migration wizard similar to the one for migrating .net code. You can use this or you can also call DEVENV from the command line with the /UPGRADE switch to migrate your code. You seem to get the normal features of the Visual Studio migration wizards. One limitation here is I believe the project migration does not change the project configurations from the old Development/Deployment modes so you will have to do this manually. And also apparently you can migrate from BizTalk 2006 or BizTalk 2006 R2 to BizTalk 2009.

Unit Tests

As I have mentioned in some other recent posts there is a new Unit Tests property as shown in the picture below. This will make the compilation process generate some additional code to help you unit test various BizTalk artefacts. If you would like some more information about how to use these features please refer to my updated BizTalk Testing Guidance on the following link:

http://geekswithblogs.net/michaelstephenson/archive/2008/12/12/127828.aspx

Improved Build Performance

Apparently when BizTalk projects are compiled now the MsBuild will be able to tell if things have changed and only compile what has changed. Maps and Schemas seem to be complied to a .cs file before being compiled into the assembly a bit like Orchestrations could be if you made that tweak in earlier versions of BizTalk. This means MsBuild is able to detect the changes to these files and only recompile them as needed. This should in theory improve the overall solution compilation time. This could make a big difference on those projects where you end up with lots of assemblies.

This is most of what I remember from the session which was really useful, thanks to the guys who put this together.

Performance Story about Traceflag T1118 and the cluster registry checkpoint service

I’ve just got back from a week working with a customer. The customer was a large retailer using BizTalk to perform processing over AS2.  They had just installed a new QA environment and wanted the operational environment to be optimized for their application.  I had a gotcha with SQL Clustering which I wasn’t aware of and I wanted to share.


As with any BizTalk installation, I started from the ground up and switched their SQL Storage from Raid 5 (which is very bad for write performance – which BizTalk does a lot of) to RAID 10.  Over time in the lab I’ve also found that changing the NTFS cluster size can gain performance improvements.  Typically we use 64KB for data LUNS (which host the BizTalk Database MDF files) and 4KB NTFS cluster size for log file LUNS (LDF’s).


Optimizing TEMPDB is another of the optimizations that we tend to do.  Specifically splitting temp db into multiple equal size files, 1 per Logical Proc available on the server.  To reduce contention further on SQL Server and prevent almost all single page allocations Trace Flag T1118 is then applied.


I used SQL 2005 configuration manager to make the change by opening the tool, clicking SQL Server 2005 Services and then double clicking SQL Server (instance name).  In the SQL Server (Instance name) properties dialog box on the advanced tab I added the following text at the end of the existing string: -T1118.  Crucially what I’d forgotten to include was the semi colon required.  So what I’d actually done was change the following


…-lc:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\mastlog.ldf


to


…-lc:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\mastlog.ldf-T1118


In other words the SQL Server would not attempt to load a log file for the Master database with name mastlog.ldf-T1118. Still unbeknown that I had made this error we rebooted the cluster so an OS hotfix could take affect. 


When the server rebooted SQL Server would not start and the following error was in the SQL Error log


Can’t open file “c:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\mastlog.ldf -T1118”


We removed the T1118 from the Configuration Manager properties used previously and attempted to bring the SQL Service back online.  It failed again with the sam error in the error log. We examined the SQL configuration manager properties and the -T1118 is still appended to the Master log file name.  We try this several times even going into the registry directly (HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLServer\Parameters) to try and change it.  But no matter what we do it stayed there.


 Those of you who are cluster gurus reading this are probably laughing because you know what has happened.  If SQL is clustered it is the WIndows Clustering Service which controls its execution and manages based on the clsuter config which node it will start on.  In order that SQL Server functions correctly when failed over between servers, the cluster service stores a checkpoint registry on the cluster quorum disk.


In this case when the change was made and the cluster was rebooted, this incorrect configuration was stored on the quorum disk. Because changes to the checkpoint can only be made when the service (in this case SQL) is online the change I was making to remove T1118 was never getting saved and the incorrect config was consistently erasing our right config.  Talk about frustrating.


Fortunately there is a work around on this, which one of my colleagues in the Clustering Support Team was able to help me out with.


 1. Disable the cluster checkpoint for the specific registry key:


c:\> cluster res “SQL Server” /removecheck: “Software\Microsoft\Microsoft SQL Server\MSSQL.\MSSQLSERVER” 


2. Correct the trace flag using SQL configuraiton manager or the registry


3. Re-enable Cluster checkpoint for the specific registry key:


c:\> cluster res “SQL Server” /addcheck: “Software\Microsoft\Microsoft SQL Server\MSSQL.1\MSSQLSERVER”

Duplicate SOAP Subscription on Dynamic Request-Response Port

Joy of joys, more subscriptions problems to debug this week with some unexpected results.
The first sign of problems were persistence errors:
Microsoft.XLANGs.Core.PersistenceException: Exception occurred when persisting state to the database. —> Microsoft.BizTalk.XLANGs.BTXEngine.PersistenceItemException: A batch item failed persistence Item-ID dbbd66a3-d748-4051-a238-fea9509efcf4 OperationType MAIO_CommitBatch Status -1061151949 ErrorInfo The message found multiple request response subscriptions. A message can only be […]

Incoming version MBV 10.00 : Query custom properties support included and Gui interface redesigned

Incoming version MBV 10.00 : Query custom properties support included and Gui interface redesigned

Hello,


 


Long time I did not write a post about MBV !  two reasons mainly : I was quite busy in my BizTalk Tech Lead job and I released until now on my blog only new minor MBV builds for which I considered that dedicated posts were not maybe justified; so I just made them available on my classic download link http://blogs.technet.com/jpierauc/pages/msgboxviewer.aspx  listing quickly their new small features and the fixes.


I had took time however to think about an interesting major feature I could include in MBV to give to the tool again more flexibility for users and more attraction, and I think it can justify a post this time to present it :-).


 


I will make available very soon (probably next week) a new major build (10.00)  containing I think an interesting new feature that I present below.
This build is not available immediately as I have small tests to do yet before and I would prefer that an important Access Denied  bug found very recently is solved definitively (see below for more details) before.


I had so noticed two main areas of improvement in MBV :


 


1)      Until now, it was not possible to enter at runtime some properties per query (like a “sort order” prop for example) . The sort order properties  for “Send Ports/RL/Orchs”  or “Tables Size” queries were well present in previous versions but were just some options in the ”Options” tab and were specific to only these queries.
I thought also that it could be interesting to be able to specify some properties at runtime for any type of query contained in MBV before to start an analyze

I decided so to add support of query custom properties in  my Health Check engine that MBV use (I will write soon another post presenting you MBV Archirtecture and the HC engine that I developed which can be used very easily in any .NET app to do a custom Health Check).

Each query can have so 0 to N custom properties with following attributes :


a.       ID
b.      Caption
c.       Default Value
d.      Value
e.      ReadOnly flag
f.        Mandatory Value flag


When a such property is defined for a query, its ID can be used in the query body or query title to be replaced dynamically during collect execution, using syntax “%u00e0 la macro” %<MYPROPID>%


This possibility allow so now to have parameterized queries with possibility then to enter quickly parameters value before to start a collect.
This new build of MBV  that I hope to release next week will allow so to enter very easily value for each custom property present for a query (if they are not read only) depending of the query selected in the Query ListViews.


 


2)  Until now, I have to admit that Gui interface of MBV tool (so the Gui tool version “MsgBoxViewer.exe” ) was quite “heavy” and confusing with lot tabs and fields everywhere present in the form  and


That’s why, with advises of Jean Severino my colleague, I redesigned the interface to be more easy to use, more intuitive, and less loaded I think :


a   . I removed the User Roles support as it was rarely used
b.      I grouped the output files Path fields and periodic collect options in the OPTIONS tab
c.       I removed the numerous Queries output Tabs keeping only two:  one displaying the Header and one displaying the Summary Report
d.      For each query you select (not just check), its standard and custom properties are displayed immediately on the right side and you can enter quickly values for custom properties.

Once I will make available this version on this blog, you could see concretly this possibility selecting for example one of these queries :   “Send Ports”, “Db Space”,  “BizTalk Perfmon counters”, “Last Svc Instance” etc.  you will see then the corresponding custom propertis displayed on the right side of the dialogbox and can change them if you want


Each value you enter in a custom property will be saved into the MBV XML Settings file when you exit the tool 


Other Features in build 10.00:



I added also optional queries below in this build suggested by Shaheer and Mike  who are Escalation engineers in Microsoft BizTalk US Support team

See their very interesting blog http://blogs.msdn.com/biztalkcpr/default.aspx !


          Get number of rows in spool associated to dehydrated instances

         
Rows/month for some tables (“Spool”, “Instances”, and some DTA tables)


 


And finally I added additional Check Rules on the optional “Last Max  100 Application Events errors” query identifying know issues from the error event entries found in your BizTalk Server Application Journal. I will add more and more warnings rules for this query in each next version, the idea is indeed to also use MBV as an “Application Journal Analyzer” identifying automatically some known important BizTalk issues when possible.


 


Fixes:



I had to fix urgently  an important Access Denied issue occurring when an MBV user is not sysadmin on BizTalk Mgmt Db as it prevent MBV to start correctly its analyze.
I found indeed that Mgmt table “adm_Messagebox” do not have permissions like “adm_group” for example and because MBV is doing a select in this table to get all the messagebox, it fails raising an Access Denied error and stop immediately.
I found finally that a Mgmt SQL stored procedure is used  instead to get that info and this sp is doing itself the select statement


 


Mark Thomas who is a famous BizTalk Field engineer at Microsoft UK reported me this problem recently and I made so a first fix but tests are still pending to see if this problem is definitively solved or not and if we don’t have other Access Denied erros later in the analyze process, that’s why I can not make available now this new build as I would prefer to publish it only if I’m sure these issues are fixed.


So, I have again to some very minor modifications and small tests do before to share publicly this version on my blog but I hope to publish it next week.
I sent it internally today to Microsoft engineers so maybe you can receive it anyway if you open a BizTalk incident at Microsoft Support


 


So monitor the blog in next days to get this new version


 


Thanks !


 


JP