BizTalk Documentor Update

BizTalk Documentor Update

Darrenhas made available a new release (v.3.2.1) of theBizTalk Documentor. It fixes some long standing issues and also some issues that came up when used with BizTalk 2009. Check it out and let us know if you run into any problems.
We are planning some re-work of this tool as well as the venerable Orchestration Profiler […]

Preserve Interchange – what is it, and why use it

The below setting is somewhat misunderstood:

If you are to set this setting, what is passed to the Message Box, and what is the behavior.

To simulate this, I created a send port with the following setting:

When I ran the 867 (found in the samples folder of the SDK) I got the following error message:

There was a failure executing the send pipeline: “Microsoft.BizTalk.DefaultPipelines.XMLTransmit, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “XML assembler” Send Port: “EDI Drop Off” URI: “C:\Users\Administrator\Documents\Visual Studio 2008\Projects\EDI\Output\%MessageID%.xml” Reason: This Assembler cannot retrieve a document specification using this type: http://schemas.microsoft.com/BizTalk/EDI/X12/2006/InterchangeXML#X12InterchangeXml.

I could not find the schema in the BizTalk 2009 folder.

Here is the data that is created by the EDI DASM with the Preserve Interchange setting:

<ins0:X12InterchangeXml DelimiterSetSerializedData="126:13:10:42:58:-1:-1:-1:-1" xmlns:ins0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006/InterchangeXML">
  <ns0:ISA xmlns:ns0="http://schemas.microsoft.com/Edi/X12ServiceSchema">
    <ISA01>00</ISA01>
    <ISA02>          </ISA02>
    <ISA03>00</ISA03>
    <ISA04>          </ISA04>
    <ISA05>ZZ</ISA05>
    <ISA06>7654321        </ISA06>
    <ISA07>ZZ</ISA07>
    <ISA08>1234567        </ISA08>
    <ISA09>991221</ISA09>
    <ISA10>1226</ISA10>
    <ISA11>U</ISA11>
    <ISA12>00401</ISA12>
    <ISA13>000000025</ISA13>
    <ISA14>0</ISA14>
    <ISA15>T</ISA15>
    <ISA16>:</ISA16>
  </ns0:ISA>
  <FunctionalGroup DocType="http://schemas.microsoft.com/BizTalk/EDI/X12/2006#X12_00401_864">
    <ns0:GS xmlns:ns0="http://schemas.microsoft.com/Edi/X12ServiceSchema">
      <GS01>TX</GS01>
      <GS02>01</GS02>
      <GS03>1234567</GS03>
      <GS04>19991221</GS04>
      <GS05>1226</GS05>
      <GS06>2</GS06>
      <GS07>X</GS07>
      <GS08>004010</GS08>
    </ns0:GS>
    <TransactionSet DocType="http://schemas.microsoft.com/BizTalk/EDI/X12/2006#X12_00401_864">
      <ns0:X12_00401_864 xmlns:ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
        <ST>
          <ST01>864</ST01>
          <ST02>3313</ST02>
        </ST>
        <ns0:BMG>
          <BMG01>00</BMG01>
        </ns0:BMG>
        <ns0:DTM>
          <DTM01>001</DTM01>
        </ns0:DTM>
        <ns0:N1Loop1>
          <ns0:N1>
            <N101>001</N101>
          </ns0:N1>
          <ns0:N2>
            <N201>N201</N201>
          </ns0:N2>
          <ns0:N3>
            <N301>N301</N301>
          </ns0:N3>
          <ns0:N4>
            <N401>N401</N401>
          </ns0:N4>
          <ns0:REF>
            <REF01>01</REF01>
            <ns0:C040>
              <C04001>01</C04001>
              <C04002>C04002</C04002>
            </ns0:C040>
          </ns0:REF>
          <ns0:PER>
            <PER01>1A</PER01>
          </ns0:PER>
        </ns0:N1Loop1>
        <ns0:MITLoop1>
          <ns0:MIT>
            <MIT01>MIT01</MIT01>
          </ns0:MIT>
          <ns0:N1Loop2>
            <ns0:N1_2>
              <N101>001</N101>
            </ns0:N1_2>
            <ns0:N2_2>
              <N201>N201</N201>
            </ns0:N2_2>
            <ns0:N3_2>
              <N301>N301</N301>
            </ns0:N3_2>
            <ns0:N4_2>
              <N401>N401</N401>
            </ns0:N4_2>
            <ns0:REF_2>
              <REF01>01</REF01>
              <ns0:C040_2>
                <C04001>01</C04001>
                <C04002>C04002</C04002>
              </ns0:C040_2>
            </ns0:REF_2>
            <ns0:PER_2>
              <PER01>1A</PER01>
            </ns0:PER_2>
          </ns0:N1Loop2>
          <ns0:MSG>
            <MSG01>MSG01</MSG01>
          </ns0:MSG>
        </ns0:MITLoop1>
        <SE>
          <SE01>18</SE01>
          <SE02>3313</SE02>
        </SE>
      </ns0:X12_00401_864>
    </TransactionSet>
    <ns0:GE xmlns:ns0="http://schemas.microsoft.com/Edi/X12ServiceSchema">
      <GE01>1</GE01>
      <GE02>2</GE02>
    </ns0:GE>
  </FunctionalGroup>
  <ns0:IEA xmlns:ns0="http://schemas.microsoft.com/Edi/X12ServiceSchema">
    <IEA01>1</IEA01>
    <IEA02>000000025</IEA02>
  </ns0:IEA>
</ins0:X12InterchangeXml>

So what can consume this message?

If you look at Preserving a Received Batched EDI Interchange, it does not explain about what happens to the document.

If you look at Sending a Preserved Batch Interchange, it will recreate the interchange based on the settings described on the page.

I asked why there is this option, to preserve interchange, getting a document that is only consumable by the EDI ASM, and was told that you can create a map that transforms all of the EDI transactions from one to another and keep the batch together, however this does not seem correct, because a map would be looking for a MessageType of http://schemas.microsoft.com/BizTalk/EDI/X12/2006#X12_00401_864, not http://schemas.microsoft.com/BizTalk/EDI/X12/2006/InterchangeXML#X12InterchangeXml

It seems that this is only for validation, or re-enveloping.

Hosting Enhancements in .NET4 and Windows Server AppFabric

This blog post drills down on the various hosting capabilities of WCF and WF in .NET 4 and for Window Server AppFabric as introduced in in the Introduction to AppFabric blog entry.


When referring to hosting, I am discussing the features of the “application environment”, which enables execution and management of the applications.


The .NET 4 for WCF Web services one focus was on making the application easier to configure and manage. For Workflow (WF) Web services, we integrated the “application environment” with WAS (Windows Process Activation Service) so that you could deploy, execute and manage WF services.


Windows Server AppFabric extends the WAS “application environment” to provide two additional hosting capabilities, namely  “Always Running” and “Scheduled Start”.


I will describe these hosting features in detail in this post, so let’s start with the improvements in .NET 4.


.NET 4 Hosting Enhancements


1)     Config Based Activation (CBA):


Today when hosting web applications containing services, you need to deploy an additional “.svc” file, describing metadata and information about the services. When organizations/applications have large number of services, there is an inherit explosion of files causing a management overhead. To address this problem, we introduced the config based activation feature which allows registration of the services in the web.config files, eliminating the need for a .svc file.


How-To:


a.     Add the service registration to the web.config file of your web application.


In the example below, we register two services, the CreditCheckService and the PaymentService. In addition the Payment/PaymentService.svc specifies a customfactory that can be used for activation.


<system.serviceModel>


 <serviceHostingEnvironment>


  <serviceActivations>


        <add relativeAddress= “CreditCheckService.svc” service=“Microsoft.ServiceModel.Samples.CreditCheckService”/>


        <add relativeAddress= “Payment/PaymentService.svc” service=“Microsoft.ServiceModel.Samples.PaymentCheckService” Factory=”CustomFactory”/>


  </serviceActivations>


 </serviceHostingEnvironment>


</system.serviceModel>


 


Note: You have the option of using either Service Config Editor or Visual Studio Intellisense to configure the application.


More details about this feature can be found here;


%u00b7       http://msdn.microsoft.com/en-us/library/ee358764(VS.100).aspx


%u00b7       http://blogs.msdn.com/rampo/archive/2009/10/27/activation-without-svc-files-config-based-activation-cba.aspx


2)     Support Multiple Site Bindings:  


Many hosters and organizations host their web application over multiple ports and hostnames. This makes their service accessible over multiple addresses. For example the following addresses resolve to the same service.


www.store.com


store.com


OR


www.store.com:80


www.store.com:90


 


The URLs above require specifying multiple site bindings in IIS. Lack of this support for this feature in the WCF runtime was an adoption blocker in the 3.0 release of the .NET framework. In .NET 3.5, we made some incremental improvements through  baseAddressPrefixFilters. In .NET 4, we have addressed more of this feature, which has significantly eased the shared hosting deployment issues.


How-To:


a.     Configure Site: Configure IIS site binding with the required host headers.


b.     Enable Application: Configure the application web.config file to enable multiple site bindings. See example below.


<system.serviceModel>


 <serviceHostingEnvironment multipleSiteBindingsEnabled=”true”>


<system.serviceModel>


 


Note: Only multiple of http/https site bindings are supported. i.e. TCP and other protocols are not currently supported


More details about this feature can be found here;


%u00b7       http://msdn.microsoft.com/en-us/library/ee358763(VS.100).aspx


%u00b7        http://blogs.msdn.com/rampo/archive/2008/02/11/how-can-wcf-support-multiple-iis-binding-specified-per-site.aspx


3)     Enable Extension-less URLs


There are scenarios when you might want to deploy your web service without extensions in the URL say for example, a WCF REST RSS service deployed with these URL


http:://www.store.com/products/


http://www.store.com/Movies


 


We leveraged System.Web.Routing feature and added a new System.ServiceModel.Activation.ServiceRoute API to enable the registration of WCF services. This enabled us to extend an API customers were already familiar with it to WCF Services.


How-To:


The example below enables activation of a WCF REST service over the URI http://www.store.com/movies


a.     Enable runAllManagedModulesForAllRequests


Add the following configuration to the application web.config file. Note: the Visual Studio web application template includes this by default.


<modules runAllManagedModulesForAllRequests=true>


b.     Register the Route Handler/Module


<add name=UrlRoutingModule


               type=System.Web.Routing.UrlRoutingModule,


                   System.Web.Routing, Version=4.0.0.0,


                   Culture=neutral,


                   PublicKeyToken=31BF3856AD364E35 />


            </modules>


      <handlers>


        <add name=UrlRoutingHandler


              preCondition=integratedMode


              verb=* path=UrlRouting.axd


              type=System.Web.HttpForbiddenHandler,


                  System.Web, Version=4.0.0.0, Culture=neutral,


                  PublicKeyToken=b03f5f7f11d50a3a />  


      </handlers>


c.      Register the service either in global.asax or AppInitiate() method deployed to App_Code directory.


protected void Application_Start(object sender, EventArgs e)


 {                 RouteTable.Routes.Add(new ServiceRoute(“movies”, new WebServiceHostFactory(), typeof(Movies)));


}


More details about this feature can be found here – 


%u00b7         http://msdn.microsoft.com/en-us/library/ee358764(VS.100).aspx


4)     Workflow in WAS


As part of providing the “application environment” for WF, we enabled


1)     Activation of declarative .xamlx Workflows when deploy to WAS


2)     Continued execution by identifing instance performing work and reporting it to the hosting environment. This ensures that the host process is not un-necessarily recycled.


How-To:


Use the WCF workflow Service Application template to develop WF application to be hosted in WAS/Windows Server AppFabric. These templates will generate .XAMLX resources which are directly hostable in WAS.


Note: if you have custom activities which need to be deployed along with the application, then they need to reference it in the system.web.compilation <assemblies> element.


Windows Server AppFabric Hosting


Now let’s look at the feature for Hosting that Windows Server AppFabric provides in addition to the .NET Framework.


5)     Windows Server AppFabric Service Management Service


There are mainly two scenarios why we implemented the service Management Service in Dublin. They are:


A    Batch processing


a.     The scenario is that you want to batch process orders at off peak times or weekend. Typically this scenario is that a web application takes the order and puts it in the queue, and the orders are processed from at the end of the day when servers are typically otherwise idling.


B    Support Windows Server AppFabric Workflow Management Service


b.     The Workflow Management Service (WMS) is responsible for resuming “runnable” durable instances. See Introduction to AppFabric blog entry for details. This service uses the Service Management Service as a mechanism to start the WorkflowServiceHost. This is an implementation detail of the Windows Server AppFabric Workflow Management Service.


To support the above requirement there needed to be a protocol agnostic way to start a service and so we introduced the Windows Server AppFabric Service Management Service, which enables starting any service (registered with the application environment) within the application.


How-To


The first question to ask is “How do I address this service?” To demonstrate this with an example.


http://www.store.com/Services/dvd.svc/titleInfo


Above is a URI  of a dvd.svc .Remember the fact that Service Management service is an independent service, which resides in every virtual application, and is capable of being activated over the named.pipe transport protocol.


It implements an IServiceManagement interface, which contains the ActivateService method, with one parameter the virtualPath.


Returning to our example, let’s say the above dvd.svc is deployed in the virtual application called “Services”.


To construct URI of the Service Management service


1)     Replace the protocol the service URI with net.pipe


2)     Replace the hostname with the appropriate host header specified in the site binding


3)     Remove the relativeServicePath and append it rest of the path with /ServiceManagement.svc


So for this example, the Service Management Service can be specified with


net.pipe://localhost/Services/ServiceManagement.svc  


Typically to start a service, you could envision a application which takes a service URI, calculates the SMS URI, and pings the application.


6)     Autostart WCF/WF Services  


One issue when hosting services in IIS is the “first message latency” message delay or the the requirement to “Warm-up” the application, in advance of the first message.


Autostart is a feature introduced in Windows 7 and Windows Server 2008 R2 which allows you to achieve this. It provides “Warm-up” and the “Always Running” semantics. Additionally, It removes the need to write a WAS host aware transports, otherwise known as Listener Adapters.


You should use this feature for primarily two reasons


a.     Reduce the latency incurred by the first message.


b.     Hosting WCF transports/protocols for which there are no Listener Adapters. The trade-off is site density


This feature is supported only on the Win7 and Win2008 R2 platform.


How-To:


Windows Server AppFabric provides configuration support in IIS Manager to Autostart the application.


1.     Navigate to the Application


2.     Click on Configure in Actions pane


3.     Configure the Application to either by choosing


a.      Enabled which will  autostart all the services


or


b.     Custom option, which will  autostart only the specific services.


 



 


To Autostart an individual service, you will need to navigate the configuration panel for that specific service.


To summarize, we made hosting investments , in both the framework and Windows Server AppFabric,  primarily focusing on the following aspects


1)     Easier WCF service configuration and management


2)     Enable activation and execution of Worflow  in WAS


3)     Enhance the existing “application environment” capabilities in WAS by providing additional semantics like “Always Running” and “Scheduled Start”  for web services.

Dutch DevDays 2010

Dutch DevDays 2010

The Microsoft DevDays Netherlands 2010 show (“Dutch DevDays”) was announced today for March 30-31, 2010 in The Hague. The Website is now live, and registration open:

http://www.devdays.nl/

The big news is they lined up Anders Hejlsberg to keynote the event.

Pluralsight is planning to have a strong presence at the event again this year. We had a wonderful time at last year’s event. Hope to see you in The Hague next year!

Accouncing the New BizTalk Message Archiving Pipeline Component

It gives me great pleasure to announce Version 1.0 of the BizTalk Message Archiving Pipeline Component and my new venture, Atomic-Scope.
The component allows you to archive Xml, flat-file, EDI and binary messages as they pass thru receive and send pipelines, without the need for complex ‘archiving’ Send Port configurations, additional load on the MessageBox database […]

New MBV builds : Lot of additional queries and fixes

New MBV builds : Lot of additional queries and fixes

Hello,


 


I updated today a new build of MBV.


With the previous one published few weeks ago, MBV brings now lot of new interesting queries and fixes (see below the complete list).


 


As interesting new query, MBV is parsing now your SQL Error logs for critical errors and raise corresponding warnings if found.


MBV is reporting now also the SQL agent log containg any errors if the BizTalk jobs are failing.


MBV includes also now the “tempdb” inf Db analyzed so you can see quickly in MBV report the size and  files info about your temp. Knowing as important the tempdb is, it can be useful to have this info in report as well.
I added also again some new parsing rules for the Eventlog query identifying so possible know issues bases on the event source, event ID, and the event message contain.


 


Feel free so to use now latest version on my blog (http://blogs.technet.com/jpierauc/pages/msgboxviewer.aspx) containing these features and report me your comment or bugs found


 


Thanks !


 


JP


 


 


New Features:


– Include “Tempdb” info  on the “Dbs infos”, “Dbs Files”, and “Dbs Space” queries
– Query to get list of IIS App pools with their owning Applications and check if an Application is a BizTalk Receive Location
– Query to get list of Web sites with all their propertes App pools with their owning Applications and check if an application is a BizTalk Receive Location
– Get list of active global SQL Trace flags
– Query to get the SQL agent log
– Changed the KB linked to the warning message when AllowOnlySecureRpcCalls  is not set correctly on cluster
– Fixed HotFix queries to get also the 32 bits installed on a X64 machine
– Display now HotFix file(s) with their version
– Fixed the .NET config files to have also the ones from 64 bits version of the Framework
– Get also BTSNTSVC64.exe.Config if found
– Order by DESC the Most recent records in TDDSFailedTRRackingData table for both DTA and BAM dbs
– Fixed an HTML display issue (report can be expanded correctly ) when we fail to connect to  some Dbs
–  Added Autogrow rules for the “DBs File” query to check the current auto grow of BizTalk Dbs against our recommendations for MDF and LOG files- New query in “Server Info” category: “NET Config files” on all BT servers – not checked by default
– New query in “Server Info”: “Running Processes” on all servers
– New query in “Advanced DB Info” category: “Tracking tables Sequence Number”  –  not checked by default
– New query in “Advanced DB Info” category:  “Tracking tables Sequence Number gaps” – not checked by default
– New query in “Advanced DB Info” category : “Tracking tables Sequence Large BLOBs” – not checked by default
– Provides a rule raising a warning if large table “NotEqualPredicateTables” found (known issue)
– Provides additional EventsLog rules
– Improves MsgBox Database naming convention in some Summary Report categories to be unique in multiple MsgBox scenarios
– Provides “Total Q rows” entry in the “MsgBox Table” Summary Report category
– “Current Error Log” query is modified to list entries in Descendant order
– Provides some rules to the “Current Error Log” query to check for Disk space errors, Db Integrity or Fatal errors, and raise warnings if   found
– “Error Log.1” request in now moved into its dedicated query  – not Checked by default
– Host instance “Start time” is now added in Topology Report “Running Host instance” category
– System Variables are added in Topology Report

Speaking in San Diego Tuesday – I’ll be doing a PDC recap from a Connected Systems perspective

I’ll be doing a presentation at the Connected Systems SIG of the San Diego .NET User Group this coming Tuesday, Dec 8th.

Meeting will be at the Microsoft La Jolla office, 6:00 for pizza, meeting starts at 6:30 and goes until we decide to stop 🙂

 

Topic

Recap and discussion of some of the major Connected Systems, SOA and ESB announcements and product roadmaps from PDC

 

Abstract

PDC saw a flurry of announcements about emerging technologies and future versions of existing ones. Join Microsoft’s Brian Loesgen as he re-caps some of the major announcements, and what they mean for people working in Connected Systems technologies, or people building SOAs. Brian will also spend some time discussing the SOA Manifesto (http://soa-manifesto.com) which he helped co-author and was released at the SOA Symposium conference in Rotterdam recently. He will also show (for the first time) a cool new demo that integrates on-premises ESB with the Azure platform AppFabric Service Bus, a new design pattern we have at our disposal (and will be in fully supported production when Azure goes live Feb 1 2010), and will be widely used.

This will be an informal discussion, with some demos and videos, and will cover a broad range of aspects of future technologies. Should be a fun meeting!

Announcing Pluralcast: Pluralsight’s .NET Podcast

Announcing Pluralcast: Pluralsight’s .NET Podcast

I’m excited to announce the latest addition to Pluralsight’s online offerings: a bi-weekly audio show about life on the Microsoft .NET stack, hosted by our very own David Starr. We’ve decided to call it the Pluralcast.

pluralcast

Available at http://cast.pluralsight.com.

This podcast is going to be different than the other ones you’re used to listening to.  Most podcasts focus on one topic and one guest.  That’s where Pluralcast differs – each Pluralcast show will include multiple guests, a plurality of perspectives and opinions, hence a plural-cast.

If you browse to the Pluralcast and click on About the Show, you’ll find the following description:

There are as many views on technology as developers who hold them. A technology implementer is going to see the world slightly different than a technology creator or expert, likewise a trainer or a student will have a different perspective of the technology from being in the field teaching and learning. The Pluralcast is about life on the Microsoft stack, from all of these perspectives, bringing together a balanced conversation to the podcast space, something we feel is missing today.

The Pluralcast is a bi-weekly audio show for the Microsoft developer community interested in hearing something new about technology. Each episode has a specific theme with selected stories, providing a holistic view around technologies developers use in the real world.

The Pluralcast is hosted by David Starr, a Pluralsight instructor with amazing yoyo skills and a passion for agile software development and the tools that support it. David has over 18 years of experience in software development and has held numerous leadership positions in technology teams. He walks the earth in search of compelling stories of software developers and their feats of daring to share with the world. Read David’s own announcement about the show and what he would like it to become for the community.

Do you have something to share with the community? Are you a technology implementer? How about a student just getting started in the Microsoft stack? Did you create a technology you’d like to talk about? Do you have mad yoyo skills? Regardless of your perspective or story, if you want to share it, tell us about it so David can connect with you. 

We just took the show live last week but the first three episodes are already available simply browse to http://cast.pluralsight.com and subscribe to the Pluralcast feed or the iTunes feed to stay connected to future shows. We hope you enjoy the Pluralcast.

My Presentations in Europe (December 2009)

My Presentations in Europe (December 2009)

This past week I’ve been traveling around Europe giving a bunch of presentations (approximately 5 hours in each country) – Norway on Tuesday, Sweden on Wednesday, Denmark on Thursday, and Belgium on Friday.  I’ll then be presenting at the BizSpark Camp in France this coming Tuesday.

Things went well with the talks, and I had fun meeting lots of new people (more than 3,000 attended the talks!). Below is a picture of my talk in Belgium – where everyone showed up in a red shirt 🙂

belgium

Download Talks

My talks were filmed in a few locations – and I’ll update this post with pointers to the videos once they are online to watch.

Below are copies of my slides + demos bits:

Hope this helps,

Scott

P.S. In addition to blogging, I’m also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu