Congratulations…

by | Aug 15, 2008 | BizTalk Community Blogs via Syndication

"The Microsoft Competency has once again walked off with the Business Processing and Integration award at Microsoft's annual Partner Summit, held last week at the International Convention Centre in Durban. Business Connexion (Pty) Ltd, stands ahead of their competitors in this area and scooped this award for a 3rd year in a row.

We were also nominated in 3 other categories. Business Process and Integration is a solution that can give great benefits to organisations that  wish to streamline their business processes,by enabling them to connect systems and use information, deliver it in the right format, to a destination either inside or outside of the organization, across functional domains at the on time without the need for human intervention.

We congratulate all the members of the Microsoft Competency, especially our Microsoft Most Value Professional's (MVP) and wish them well, and growth for this financial year."

Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

Using the SAP ADO.NET Provider From SSRS 2005

by | Aug 14, 2008 | BizTalk Community Blogs via Syndication

One of the new features which we’re supporting in the BizTalk Adapter Pack V2 is the ability to use the SAP ADO.NET Provider from SSRS. In this post, I’ll briefly outline the steps you need to perform in order to get this to work.

Firstly, install CTP3 of the WCF LOB Adapter SDK V1 SP2, and the BizTalk Adapter Pack V2 (making sure that you install the SAP ADO Provider).

Next, you need to make a few changes to the SSRS related config files, in order to have the provider show up in SSRS projects:

  • Modify the RSReportDesigner.config file under the Visual Studio 2005 installation directory (on my machine, this is present at C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies).

In the above screenshot, the entries with the name “SAP-AP” are the ones added by me. The entries I added were:

<Extension Name=”SAP-AP”
           Type=”Microsoft.Data.SAPClient.ReportingServicesExtension.SAPConnectionWrapper,
           Microsoft.Data.SAPClient.ReportingServicesExtension,
           Version=3.5.0.0,
           Culture=neutral,
           PublicKeyToken=31bf3856ad364e35″/>

and

<Extension Name=”SAP-AP” Type=”Microsoft.ReportingServices.QueryDesigners.GenericQueryDesigner,
           Microsoft.ReportingServices.QueryDesigners”/>

  • Modify the RSReportServer.config file under the SQL 2005 installation directory (on my machine, this is present at C:\Program Files\Microsoft SQL Server\MSSQL.2\Reporting Services\ReportServer).

In the above screenshot, the entry with the name “SAP-AP” is the one added by me. The entry I added was:

<Extension Name=”SAP-AP”
           Type=”Microsoft.Data.SAPClient.ReportingServicesExtension.SAPConnectionWrapper,
           Microsoft.Data.SAPClient.ReportingServicesExtension,
           Version=3.5.0.0,
           Culture=neutral,
           PublicKeyToken=31bf3856ad364e35″/>

 

We’ll now create a simple SSRS project using the “SAP-AP” provider.

  • Start Visual Studio 2005, create a New Project using the template under Business Intelligence Projects -> Report Server Project. Click OK in the dialog below.
  • Right click on the “Shared Data Sources” node, choose “Add New Data Source”.
  • Create a new data source named “SAPDataSource”, as shown in the screenshot below:
  • In the above dialog, in the Credentials tab, make sure that “No Credentials” is selected (since we’re entering the credentials directly in the Connection String). Click OK to save the information and close the dialog.
  • Right click on the “Reports” node, and choose “Add New Report”. Choose the “SAPDataSource” entry from the “Shared Data Source” drop down list. Click Next to move to the next dialog.
  • Enter the query to use in the Query String section. Click Next a few more times (the dialogs after this allow you to customize the layout – for now, you can stick with the defaults).
  • Right click on the project node in Solution Explorer, to set the start-up report. In the screenshot below, I’ve set the “StartItem” to the newly created report above (named Report1.rdl). (Note – I had named my project “Report Project1DEL”). Click OK to close the dialog.
  • Hit F5 to run the Project. A screen similar to the one below should appear. Note that there is an empty text box labeled “P1” below – this is where you would enter the value to use for the P1 parameter, in our query which we used earlier (“SELECT TOP 10 KUNNR, NAME1 FROM KNA1 WHERE NAME1 LIKE @P1”). I’ve entered the string “AB%” – which thus means that I want the first 10 customer numbers and names from the KNA1 table, where the customer name starts with “AB”.
  • Click the “View Report” button (top-right corner) to execute the query and view the data. On my machine (running against my SAP server), I see the data below:

 

Question – do you have a need to use the Siebel ADO.NET Provider from SSRS? If so, please leave a comment here (leave your email address too, so that we can contact you in case we need to understand your scenario more, etc).

Using Team Foundation Build for BizTalk projects (Part 2)

by | Aug 14, 2008 | BizTalk Community Blogs via Syndication

In my last post I wrote about the requirements and challenges we encountered, implementing Team Foundation Server to create an automated build process. This post is meant to leave a more detailed view. 

Design Code structure

The main concern when it comes to design was to make deployment as easy as possible.  To enable this, we separated all BizTalk projects by types of artifacts. That left us with three different types of projects Orchestrations, Transformation and Schemas. No reason for Pipeline projects since we created generic pipelines to be configured at runtime. This way we got a chain of dependencies from Orchestrations -> Transformations -> Schemas. This would prove very important upon automating the undeploying of these assemblies.

We used the notion of BizTalk Applications to represent actual systems from which we'd receive and send messages to. The Application name would be used in the name of the BizTalk project, together with the information type. Eg. SAP.SalesRevenue.Orchestrations.proj.

Every BizTalk project where then grouped into a ILS solution, which in this case could be GeneralLedger.sln. To make sure each ILS solution could be built and deployed independent of other solutions, there could never be any references between these solutions.

Custom build tasks

Custom build tasks extends Team Foundation Build, and is necessary when you'd want to perform actions part from what is included in the product. There are lots of custom tasks, made available through the community, such as Microsoft.Sdc.Tasks. This library has lots of nice features for many of the products from MS, among them BizTalk.

Unfortunately, we couldn't quite get them to work with our requirements. But they where much helpful as a starting point.

We ended up creating four custom build tasks:

UndeployBindings

Undeploys ports and orchestrations from BizTalk applications using binding files. For this to work, the name of the binding file(s) are expected to end with target environment, Eg Sap.BindingFile.BUILD.xml.

<VIPN.Shared.Tools.BizTalkTfsBuildTasks.UndeployBindings
                TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                BuildUri="$(BuildUri)"
                Server="$(BizTalkServerName)"
                RootPath="%(SolutionToBuild.Identity)"
                Database="$(BTServerDatabase)"
                ProcessTimeOutMilliseconds="60000"
                TargetEnviroment="DEV"/>
UndeployAssembly

The UndeployAssembly Task is only used internally from the UndeployBinding Task,  and is used for undeploying a single assembly from an BizTalk application. This is done after all ports and orchestrations are removed. The resources needs to be removed in the right order, assemblies, with references to others needs to be removed first. This is accomplished by encapsulating schemas, transformations and orchestrations in separate projects, and naming them thereafter. Eg. [Application].[InfomationType].Transfomations

ImportBindings

If the solution path includes a "Bindings" folder, all binding files will be imported into the application. The target environment will be set, if included in the name of the binding file. Eg Sap.BindingFile.TEST.xml.

Usages:

<VIPN.Shared.Tools.BizTalkTfsBuildTasks.ImportBindings
                TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                BuildUri="$(BuildUri)"
                Server="$(BizTalkServerName)"
                Database="$(BTServerDatabase)"
                TargetEnviroment="$(TargetEnviroment)"
                RootPath="%(SolutionToBuild.Identity)"/>
ExportPackage

The ExportPackage exports one package per BizTalk Application, but only the assemblies stated in the binding file(  + all binding files for that application).

Usages:

<VIPN.Shared.Tools.BizTalkTfsBuildTasks.ExportApplication
                        TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                        BuildUri="$(BuildUri)"
                        Server="$(BizTalkServerName)"
                        RootPath="%(SolutionToBuild.Identity)"
                        Database="$(BTServerDatabase)"
                        TargetEnviroment="$(TargetEnviroment)"
                        OutputPath="$(OutDir)"/>

Download the task library, and make sure it builds. You might have to reset some of the Team Foundation references. Copy all the assemblies to a directory on the build server.

Team Foundation Build Type

To make use of all this, create a new Build type, and open the TfsBuild.proj file.  Copy the lower part of this template ( from <!– Blogical configuration –> and down). You need to change some of the configuration in the <!– BizTalk Variables –> property group.

  • <Blogical_CustomTasks> should be set to the directory on the build server where you copied the task library.
  • <TargetEnviroment> should be set to the name of your target environment, and correlate to the binding file name to use.
  • <BizTalkServerName> is the name of the database server.

That's it.

Summery

I realize that this is a complex solution, and it may not fit within your design. But hopefully it will give you some inspiration of how to solve the problem.

 

 

 

 

 

 

Surfacing BizTalk Adapter Pack Adapters (WCF Bindings) as native BizTalk Adapters

by | Aug 14, 2008 | BizTalk Community Blogs via Syndication

One of the features we’ve enabled in the CTP3 of the BizTalk Adapter Pack V2 (which requires CTP3 of the WCF LOB Adapter SDK SP2), is the ability to expose the Adapters (which are in reality, WCF Bindings) as native BizTalk Adapters. Prior to this, if you wanted to use the Bindings from BizTalk, you had to use the WCF-Custom Adapter in BizTalk, then select the WCF Binding, and follow that route.

With CTP3, there is now an additional way you can use the adapters.

  • Firstly, install the CTP3 of the WCF LOB Adapter SDK SP2, and the CTP3 of the BizTalk Adapter Pack V2.
  • Open the BizTalk Server 2006 Administration Console
  • Navigate to the node as displayed in the screen shot below, right click in the empty space, and choose New->Adapter. A new dialog opens up.
  • Assuming you have the SQL and/or SAP Bindings installed during the installation of the Adapter Pack, you should see one or two entries in the “Adapter” drop down list, “WCF-SQL” and “WCF-SAP”. For the purpose of this blog illustration, choose WCF-SAP. Enter “WCF-SAP” in the Name text box, and press OK twice to close the dialog.
  • In the administration console, navigate to an existing application, and create a new Static Solicit-Response Send Port. You should now see WCF-SAP appear in the drop down list containing available Transport Types.
  • Click Configure to configure the transport. A new dialog opens up, containing a number of property pages.
  • The “General” tab looks similar to what you would see had you chosen WCF-Custom as the transport type. However, there is one addition. There is a “Configure” button above the text box where you enter the URI. You can click it to bring up a Connection Uri Builder Dialog.

  • If you navigate to the binding tab, you see that you no longer need to choose a binding. Also, the binding properties are displayed in a more user-friendly manner, with descriptions of the properties appearing at the bottom of the property grid.
  • The other four tabs (Behavior, Credentials, Messages, Import/Export) are the same as those which appear in the WCF-Custom adapter.

In CTP3, the above is available only for the SQL and SAP adapters. For the next CTP, we plan to do this for the OracleDB and the OracleEBS adapters also. Question – do you want this for the Siebel adapter too? If so, leave a comment here.

Also, in CTP3, this is being done only for BizTalk 2006 R2. For the next CTP, we plan to have this for BizTalk 2006 R3 also.

Orcas SP1 Improvement: Asynchronous WCF HTTP Module/Handler for IIS7 for Better Server Scalability

by | Aug 13, 2008 | BizTalk Community Blogs via Syndication

(Copied from https://blogs.msdn.com/wenlong/archive/2008/08/13/orcas-sp1-improvement-asynchronous-wcf-http-module-handler-for-iis7-for-better-server-scalability.aspx)


Introduction


As mentioned in my last blog entry, for IIS-hosted WCF services, WCF holds the worker thread coming from ASP.NET until the whole request is completed to avoid a Denial of Service (DOS) attack. I also mentioned that on Windows 2008 Server, IIS7 has introduced the following registry setting to provide request throttling for all incoming requests:


[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ASP.NET\2.0.50727.0]


“MaxConcurrentRequestsPerCpu”=dword:0000000c


Based on this, in Orcas SP1 (.NET 3.5 SP1), WCF has implemented the asynchronous HTTP Module/Handler to allow better server scalability for high latency requests.


Asynchronous WCF HTTP Module/Handler


Besides the existing synchronous WCF HTTP Module/Handler types, WCF introduced the asynchronous versions. Together, WCF has the following four types implemented:


%u00b7        Synchronous Module:


System.ServiceModel.Activation.HttpModule, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089


%u00b7        Asynchronous Module (new):


System.ServiceModel.Activation.ServiceHttpModule, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089


%u00b7        Synchronous Handler:


System.ServiceModel.Activation.HttpHandler, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089


%u00b7        Asynchronous Handler (new):


System.ServiceModel.Activation.ServiceHttpHandlerFactory, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089


Asynchronous HTTP Module


The new type ServiceHttpModule implements the System.Web.IHttpModule and it registers the async event HttpApplication.AddOnPostAuthenticateRequestAsync with a pair of async handlers:


static public IAsyncResult BeginProcessRequest(object sender, EventArgs e, AsyncCallback cb, object extraData)


static public void EndProcessRequest(IAsyncResult ar)


When a request comes in, WCF does some validation and returns from BeginProcessRequest immediately after passing the request up to internal processing logic. In this way, the worker process is immediately released.


If the service operation is asynchronous and it is waiting for slow I/O operations, the whole stack does not hold any thread. The only performance concern would be the memory usage to hold objects used to process the request. This allows many concurrent requests to be served by WCF without using a lot of threads and thus greatly improves the scalability of the service. This is very helpful when completing each request takes significant amount of time and many clients are served. In some private testing, WCF can easily support more than 10K concurrent slow requests, which is not possible with the synchronous HttpModule.


Asynchronous HTTP Handler Factory


You might have already noticed that WCF implements the Handler Factory instead of the asynchronous Handler directly. This allows more flexibility in the future to support different types of  HTTP Handlers. The ServiceHttpHandlerFactory creates a stateless asynchronous HTTP handler of the following type which implements System.Web.IHttpAsyncHandler:


System.ServiceModel.Activation.ServiceHttpHandler


It implements the pair of asynchronous request handlers:


public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)


public void EndProcessRequest(IAsyncResult result)


Default Installation – Synchronous Version


Though the asynchronous version of HTTP Module/Handler is added in this release, the default installation is still the synchronous version. Why is this?


First of all, you should not use asynchronous version of HTTP Module/Handler on Windows 2003 Server (with IIS6) without knowing the risk. If you have many clients, the service would be easily DOS attacked by too much memory usage due to huge amount of pending requests queued up in WCF transport layer without the throttling inside IIS/ASP.NET.


Because of the limitation on IIS6, WCF did not change the setup to switch to purely asynchronous due to the ramifications between IIS6 and IIS7 for this quick release. This means that in order to use the asynchronous version, you have to perform some manual registration.


Don’t worry, I have provided a private tool to do this registration for you.


WCF Module/Handler Registration Tool


You can find the simple tool called WcfAsyncWebUtil.exe attached in this blog. Here is the usage:


Usage: WcfAsyncWebUtil <options>


Options:


  /is


    Install WCF Synchronous HTTP module and handler into IIS7 configuration


    in Integrated Mode.


  /ia


    Install WCF Asynchronous HTTP module and handler into IIS7 configuration


    in Integrated Mode.


  /l


    List WCF HTTP module and handler registered into IIS7 configuration in


    Integrated Mode.


  /t [ <throttle> ]


    Configure the throttling registry MaxConcurrentRequestsPerCpu for the ASP.NET


    integrated pipeline to the value <throttle>. Default is 100. This argument is


used together with /ia only


For example, if you want to register asynchronous HTTP Module/Handler with the ASP.NET throttling to be 1000, you can run the following command:


WcfAsyncWebUtil.exe /ia /t 1000


To list the installed WCF HTTP Module/Handler, you can run:


WcfAsyncWebUtil.exe /l


 

WCF Request Throttling and Server Scalability

by | Aug 13, 2008 | BizTalk Community Blogs via Syndication

(Copied from http://blogs.msdn.com/wenlong/archive/2008/04/21/wcf-request-throttling-and-server-scalability.aspx)


Two Threads per Request


In .NET 3.0 and 3.5, there is a special behavior that you would observe for IIS-hosted WCF services. Whenever a request comes in, the system would use two threads to process the request:


%u00b7           One thread is the CLR ThreadPool thread which is the worker thread that comes from ASP.NET.


%u00b7           Another thread is an I/O thread that is managed by the WCF IOThreadScheduler (actually created by ThreadPool.UnsafeQueueNativeOverlapped).


When you have a high latency request, you would see the following callstack in the debugger before the request is completed (with  .NET 3.5):


0ee6ee8c 5094dad5 mscorlib_ni!System.Threading.WaitHandle.WaitOne()+0xa


0ee6ee8c 50951e3b System_ServiceModel_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.ExecuteSynchronous(System.Web.HttpApplication, Boolean)+0x8d


0ee6eeb4 65fe626d System_ServiceModel_ni!System.ServiceModel.Activation.HttpModule.ProcessRequest(System.Object, System.EventArgs)+0x143


0214e4f4 65fe3fd1 System_Web_ni!System.Web.HttpApplication+SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()+0x5d


0ee6ef08 65fe804f System_Web_ni!System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)+0x41


0ee6efb8 65fe4df2 System_Web_ni!System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)+0x63b


060f05dc 66003a92 System_Web_ni!System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback)+0x56


0ee6f040 65fd8022 System_Web_ni!System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)+0x352


0ee6f0c4 65fd7e07 System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)+0x1f2


0ee6f0f4 01782374 System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)+0x1b


0ee6f138 6a2abf3c webengine!MgdGetCurrentNotification+0x236


0ee6f19c 74a22ea0 webengine!MgdGetPreloadedSize+0x4d


0ee6f1b0 74a23696 iiscore+0x2ea0


0ee6f560 6a2ac222 webengine!MgdCanDisposeManagedContext+0xb5


0ee6f570 015f1311 webengine!MgdIndicateCompletion+0x22


System_Web_ni!System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)+0x289


 


This is the worker thread that comes from ASP.NET. Another thread is still processing the request in WCF:


01fbf0ac 0e9001c7 App_Web_rsfhd6l1!HelloWorld.SimpleService.Hello(System.String)+0x14


01fbf0ac 50b8d90b System_ServiceModel_ni!DynamicClass.SyncInvokeHello(System.Object, System.Object[], System.Object[])+0x3f


01fbf0ac 50b6d245 System_ServiceModel_ni!System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(System.Object, System.Object[], System.Object[] ByRef)+0x1fb


01fbf108 509137ad System_ServiceModel_ni!System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0xd5


01fbf148 509136a6 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0xad


01fbf174 50913613 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x76


062b75a8 50913459 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x33


062b75a8 50912257 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x59


062b75a8 50911f8f System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(System.ServiceModel.Dispatcher.MessageRpc ByRef)+0x127


01fbf1f0 509115ff System_ServiceModel_ni!System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean)+0xdf


01fbf394 5090f8c9 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext)+0x1ff


01fbf3dc 5090f35e System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext)+0x169


01fbf42c 5090f2f1 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)+0x5e


01fbf42c 50232d68 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(System.IAsyncResult)+0x41


01fbf42c 50904501 SMDiagnostics_ni!System.ServiceModel.Diagnostics.Utility+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+0x28


01fbf468 50992b36 System_ServiceModel_ni!System.ServiceModel.AsyncResult.Complete(Boolean)+0xb1


01fbf4d8 50992215 System_ServiceModel_ni!System.ServiceModel.Channels.InputQueue`1+AsyncQueueReader[[System.__Canon, mscorlib]].Set(Item<System.__Canon>)+0x46


01fbf4d8 50991ffb System_ServiceModel_ni!System.ServiceModel.Channels.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(Item<System.__Canon>, Boolean)+0x1f5


00000000 5091d7e5 System_ServiceModel_ni!System.ServiceModel.Channels.InputQueue`1[[System.__Canon, mscorlib]].EnqueueAndDispatch(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean)+0x6b


00000001 50977b7e System_ServiceModel_ni!System.ServiceModel.Channels.SingletonChannelAcceptor`3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Enqueue(System.__Canon, System.ServiceModel.Channels.ItemDequeuedCallback, Boolean)+0x75


01fbf570 5094f396 System_ServiceModel_ni!System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(System.ServiceModel.Channels.HttpRequestContext, System.ServiceModel.Channels.ItemDequeuedCallback)+0x1ce


01fbf5b8 5094e4cf System_ServiceModel_ni!System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(System.ServiceModel.Activation.HostedHttpRequestAsyncResult)+0xc6


0216803c 5094defd System_ServiceModel_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()+0xdb


01fbf60c 5094dea5 System_ServiceModel_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()+0x19


01fbf638 50903be6 System_ServiceModel_ni!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(System.Object)+0x29


01fbf674 50903b26 System_ServiceModel_ni!System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper+WorkItem.Invoke2()+0x36


01fbf688 50903ab5 System_ServiceModel_ni!System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper+WorkItem.Invoke()+0x4a


01fbf6bc 5090390f System_ServiceModel_ni!System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper.ProcessCallbacks()+0x185


01fbf6e8 5090388b System_ServiceModel_ni!System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper.CompletionCallback(System.Object)+0x6f


01fbf720 50232e1f System_ServiceModel_ni!System.ServiceModel.Channels.IOThreadScheduler+CriticalHelper+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0xf


01fbf720 79405534 SMDiagnostics_ni!System.ServiceModel.Diagnostics.Utility+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x2f


01fbf744 79e7c74b mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x60


01fbf758 79e7c6cc mscorwks!LogHelp_TerminateOnAssert+0x3433


01fbf7d8 79f00eca mscorwks!LogHelp_TerminateOnAssert+0x33b4


01fbf7f8 79f00e75 mscorwks!GetCLRFunction+0x1de16


 


Why is this? Is this a scalability issue for WCF? I will explain this below.


Asynchronous Request Processing


One of the most significant pieces for WCF is that it is highly asynchronous from bottom to top for the whole channel stack and the programming model. The advantage is apparent: applications have high flexibility by issuing operations asynchronously, doing some other stuff in parallel, and coming back to handle the completed operations once they are completed. This is also scalable when the asynchronous operations are I/O bound so that you do not need to hold extra threads to wait for operation completions.


ASP.NET also supports asynchronous request processing mechanisms through IHttpModule and IHttpAsyncHandler. With these interfaces, ASP.NET can push many concurrent requests up to the layers which have such asynchronous implementations. In this way, the whole stack need a small number of threads to process a large number of concurrent requests given appropriate queuing for the requests.


However, this also exposes a memory problem when this is on IIS 6.0. When there is unlimited number of incoming requests, all of the requests would be pushed up by ASP.NET to upper layers and it would cause significant memory consumption. Thus it would cause the server to be unavailable to do its work correctly.


So in the release of .Net 3.0 and 3.5, WCF implemented synchronous versions of HTTP module and handler instead of asynchronous ones. See the following for more details.


ASP.NET Threading on IIS 6.0


When ASP.NET is hosted in IIS 6.0, requests are handed over to ASP.NET on IIS I/O threads. ASP.NET uses CLR ThreadPool worker threads to handle requests. The CLR ThreadPool automatically adjusts the number of threads based on the server workload. In order to prevent from using too many threads or using too much memory, ASP.NET has a limit on the number of threads concurrently executing requests. This is controlled by the httpRuntime/minFreeThreads and httpRuntime/minLocalRequestFreeThreads settings. If the limit is exceeded, the request is queued in the application-level queue, and executed later when the concurrency falls back down below the limit. Thomas Marquardt has written an excellent blog on this:


http://blogs.msdn.com/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx (TM0707)


In .NET 2.0, the magic config “processModel/autoConfig” in machine.config automatically configures the ASP.NET threading settings for most applications. For those which require special settings need to be configured differently. The following KB article was provided as the general guidance:


http://support.microsoft.com/kb/821268 (KB821268)


However, there is no throttling for concurrent requests in ASP.NET on IIS 6.0.


WCF Request Throttling and Tuning


Because of lacking the throttling for concurrent requests in ASP.NET, WCF would get unlimited concurrent requests pushed up from ASP.NET if WCF had implemented asynchronous HTTP module/handler on IIS 6.0. It would cause severe memory usage problem. Because of this concern, WCF implemented the synchronous HTTP module/handler in 3.0 and 3.5 instead.


Here I only describe how the HTTP module works for WCF. The logic for the HTTP handler is similar. When a request comes in from ASP.NET, WCF grabs the request if it is a WCF request (by checking the registered BuildProvider of the extension to see whether it is WCF service BuildProvider). After that, it immediately switches to a new I/O thread and hold the ASP.NET worker thread until the request is completed. You may ask, why doesn’t WCF reuse the ASP.NET worker thread to handle the request until it is completed? Here are the reasons:


%u00b7        As long as the ASP.NET worker thread is not returned back to ASP.NET, the request is not able to be completed. The client would be blocked.


%u00b7        When the WCF operation is one-way, the request has to be completed before the service operation is invoked. So the ASP.NET worker thread has to return earlier.


%u00b7        WCF supports people to complete a two-way operation earlier in a service operation when you call RequestContext.Close.


All of these require that WCF should not hold the ASP.NET worker thread once the request is claimed to be “completed”. At the same time, WCF has to hold the ASP.NET worker thread in a waited state before the request is completed due to the requirement of synchronous request processing logic. WCF relies on this ASP.NET thread throttling logic to limit concurrent requests.


Intuitively this seems to have scalability issue. Fortunately for most WCF services, especially high-throughput ones, this has quite low impact as long as the server is busy processing requests. This is demonstrated as the CPU usage of the server.


On multi-proc/multi-core servers, the default limit of concurrent requests may not be enough for the server to achieve best throughput. This would require larger limits for ASP.NET worker threads to be used to handle requests.


Throttling Tuning


In order to achieve high throughput on high-end servers, you would need to make the following throttling tuning:


%u00b7        Increase ASP.NET thread throttling to allow more concurrent worker threads to handle requests


%u00b7        Increase WCF service throttling to allow more concurrent requests


The latter reflects the change of the known WCF throttling settings such as MaxConcurrentCalls, MaxConcurrentInstances, and MaxConcurrentSessions. The former would require fine tuning of ASP.NET thread throttling as documented in the KB article KB821268 mentioned above. Basically ASP.NET does not execute more than the following number of concurrent requests:


(maxWorkerThreads*number of CPUs)-minFreeThreads


This is 12 by default on a single CPU machine. In order to get more threads to process requests, you need to increase maxWorkerThreads. Once you can get CPU fully loaded with the settings, setting “minWorkerThreads” to 2 or 3 would allow WCF to have best throughput. Here is sample setting in machine.config:


<processModel autoConfig=”false” maxWorkerThreads=”500″ maxIoThreads=”500″ minWorkerThreads=”2”/>


<httpRuntime minFreeThreads=”250″ minLocalRequestFreeThreads=”250″/>


The above settings work for most WCF applications. However, when the service has very slow operations which are I/O bound (for example, they further connects to backend database or network or file system to perform high-latency operations), you would need to use many outstanding ASP.NET worker threads to achieve parallel requests. This is bad when the number of requests is huge, for example, several thousands or more. Each thread would hold a fair amount of system resource, i.e., memory. It would cause the service to be very slow to respond. This really requires a real asynchronous design to solve the problem. Because of this, ASP.NET introduced a new throttling logic for concurrent requests in IIS 7.0 to support this scenario for WCF.


ASP.NET Throttling on IIS 7.0 in Integrated Mode


As Thomas mentioned in his blog entry (TM0707) mentioned above, the following registry value is introduced to throttle concurrent requests:


[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ASP.NET\2.0.50727.0]


“MaxConcurrentRequestsPerCpu”=dword:0000000c


The default value is 12 per CPU which means is similar as the default setting for ASP.NET worker threads. This was introduced in ASP.NET 2.0 SP1 (shipped with .NET 3.5) and thus it is available in Windows 2008. It only applies to the IIS 7.0 Integrated mode, which is exactly where WCF is registered on Windows 2008.


Scalable Solutions for High Latency Services


Though the above throttling support was added to Windows 2008, WCF still uses synchronous module/handler instead of asynchronous ones in the released versions due to timing constraint. However, the internal WCF design does allow asynchronous request processing and it is just one step away from the real asynchronous logic. Here I provide a simple wrapper for that.


Custom HttpModule Using Reflection


The internal WCF types support asynchronous HTTP module. Since they are internal, we have to rely on reflection to create a real one.


First of all, you would need to register the async events for your HttpModule as following:


            context.AddOnPostAuthenticateRequestAsync(


                MyHttpModule.beginEventHandler,


                MyHttpModule.endEventHandler);


This is registered to the PostAuthenticateRequest for the same reasons as the default WCF HttpModule.


Secondly, you would need to use reflection to create the instance of the internal WCF type System.ServiceModel.Activation.HostedHttpRequestAsyncResult:


Type hostedHttpRequestAsyncResultType = typeof(ServiceHostingEnvironment).Assembly.GetType(“System.ServiceModel.Activation.HostedHttpRequestAsyncResult”);


ConstructorInfo hostedHttpRequestAsyncResult = hostedHttpRequestAsyncResultType.GetConstructor(BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Public,


null, new Type[] { typeof(HttpApplication), typeof(bool), typeof(AsyncCallback), typeof(object) }, null);


return (IAsyncResult)hostedHttpRequestAsyncResult.Invoke(new object[] { application, false, cb, extraData });


Once you have the custom HttpModule, you can replace it with the default one that is registered in the WAS configuration %windir%\system32\inetsrv\config\applicationhost.config:


  <system.webServer>


    <modules>


      <add name=“ServiceModel“ type=“System.ServiceModel.Activation.HttpModule, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089“ preCondition=“managedHandler“ />


    <modules>


  <system.webServer>


To do that, you can also define local settings in web.config:


  <system.webServer>


    <modules>


      <remove name=“ServiceModel“/>


      <add name=“MyAsyncWCFHttpModule“ type=“AsyncWcfModule.MyHttpModule, AsyncWcfModule“ preCondition=“managedHandler“ />


    </modules>


    <validation validateIntegratedModeConfiguration=“false“ />


  </system.webServer>


The sample code is attached.


Improvement in .NET 3.5 SP1


In .NET 3.5 SP1, we will provide the real asynchronous HTTP module/handler for WCF to better solve this problem.


 

OK, Where is the new SQL Server 2008 Report Designer/Builder?

OK, Where is the new SQL Server 2008 Report Designer/Builder?

by | Aug 13, 2008 | BizTalk Community Blogs via Syndication

With SQL Server 2008 just released.  I was keen to see how the new standalone Report Designer (Report Builder 2.0) turned out.  Well, I found out it’s not released with the RTM!  It will be available for download sometime later.  To find out more about the release plan of Report Builder 2.0 have a look here. 


At first glance, it appears that the RC0 version of Report Builder 2.0 still works with SQL Server 2008 RTM.  Search for ReportBuilder.msi and you might find an old copy of RC0 floating around.