BizTalk Server tips and tricks for Administrators: Trying to annoy Tord – Moving an Event Source to a Different/Custom Windows Event Log

BizTalk Server tips and tricks for Administrators: Trying to annoy Tord – Moving an Event Source to a Different/Custom Windows Event Log

Probably this was one of the most talked and funny tips and I completely forgot to publish in my blog despite the resources despite the resources are already available for download and referenced in the session slides that you can found here. If you are familiarly with the BizTalk Innovation Day or BizTalk Summit’s, you […]
Blog Post by: Sandro Pereira

BizTalk 2013 SharePoint Adapter not respecting SharePoint 2013 View Name

BizTalk 2013 SharePoint Adapter not respecting SharePoint 2013 View Name

 

I have done a lot of BizTalk-SharePoint Integration in the past and ran into a situation recently that surprised me. There wasn’t an easily identifiable resolution online so I have decided to document this for the benefit of others.

Background

We have a process that requires a user to approve a financial summary document in SharePoint.  Once the document has been approved, BizTalk will then fetch the details behind those financial transactions, from another source system, and send them to SAP.

In the past I have leveraged SharePoint views as a way for BizTalk to pickup messages from a SharePoint document library. The way to achieve this is to rely upon meta data that can be populated within a SharePoint document library column.

Adding a custom column to a document is very simple.  Under the library tab we will discover the Create Column label. We can simply click this button and then add a column and related properties as required. 

image

With our custom column created, we can now create a view for BizTalk to “watch”.  In our example we were dealing with an approval workflow.  We can create our custom column called Status and then when BizTalk initially publishes this financial summary document(for users to approve), we can use the SharePoint adapter to populate this column with a value of Pending.  After a user has reviewed the document, that Status value can be changed to Approved.

Since we don’t want BizTalk to move Pending documents we will create a view that will only show Approved documents.  To create a custom View we can once again click on the Library tab and then click on Create View

 

image

For our purposes a Standard View will be sufficient.

image

We need to provide a View Name and can also indicate where we want this column to be positioned.

Tip – In my experience I have experienced odd behavior with spaces in the name of SharePoint entities.  My advice is to avoid spaces in names where possible.

image

Lastly, since we only want Approved documents to show up in this field we need to add a filter.

Within our filter we want to Show items only when the following is true:

Status is equal to Approved

image

We can now save our view and test it. To test it we will upload two documents.  One will have the Status of Approved and the other will have a Status of Pending.  When we click on All Documents we should see both documents.

image

When we click on our view for BizTalk, which in this case is called BizTalkMoveView we will only see our Approved document.

image

From a SharePoint perspective we are good and we can now create our SharePoint Receive Location in BizTalk.  For the purposes of this blog post I am using a Send Port Subscription; I will receive the message from SharePoint and then send it to a File folder.

In our BizTalk Receive Location configuration we are going to use the Client OM which in this case is the SharePoint Client Object API.  This allows us to communicate with SharePoint without having to install any agents on a SharePoint Server.

We also need to configure our SharePoint Site URL, Source Document Library URL and View Name

image

When we enable our Send Port and Receive Location we should receive 1 file in our File Folder right? WRONG! Both files were picked up and moved to our file folder even though we have a View enabled.

image

If we go back to SharePoint we will discover both documents are gone.

image

Issue

The issue is that for some reason, BizTalk 2013 is not using/respecting the View Name property that is available in the Receive Location Configuration.

Resolution

The resolution is to install BizTalk 2013 CU 2. The download and more details about CU2 can be found here.

Before you install, the recommended approach from Microsoft is:

  • Stop all host instances
  • Stop SQL Server Agent which is responsible for running the internal BizTalk jobs
  • Perform a Database Backup

Running the CU2 exe is pretty straight forward and only takes a couple minutes.  I wasn’t prompted for a reboot but decided to go that route regardless.

After applying the CU, I uploaded two documents again.  One had a Status  of Approved while the other had a Status  of Pending.

image

Our BizTalkMoveView is also displaying our file correctly

image

When we enable our Receive Location we will discover that only our Approved file has moved.

image

image

Our document that was in a Pending state remains in SharePoint as expected.

image

Conclusion

BizTalk 2013 was the first version that had support for the SharePoint Client Object model.  So I am not sure if this bug only impacts when you are using the Client OM within the BizTalk Receive Location.  I do know that in previous versions of BizTalk that this was not an issue.  However those versions of BizTalk relied upon the SharePoint Adapter Service being installed on the remote SharePoint Server.  Using the Client OM is the way to go as it also allows you to communicate with SharePoint Online/Office365.

Collection of Microsoft Integration Stencils for Visio 2013

Collection of Microsoft Integration Stencils for Visio 2013

In my two previous editions (here and here) I published some shapes to represent BizTalk Server physical architectures, Integration architectures (Cloud or Hybrid scenarios) or solutions. In this resource I change the name of my Visio 2013 Stencils resource, now is called “Microsoft Integration Stencils for Visio 2013”, to include almost the Microsoft Integration Stack. […]
Blog Post by: Sandro Pereira

BizTalk Server tips and tricks for developers: How to easily set the connection string in the database lookup functoid

BizTalk Server tips and tricks for developers: How to easily set the connection string in the database lookup functoid

As most of you already know, last month I presented a session at BizTalk Summit 2015 London event about “BizTalk Server tips and tricks for developers and admins” (You can check the video recording of my session here) It was a lightweight session (30 minutes) about useful tips that we can use in our daily […]
Blog Post by: Sandro Pereira

My session about “BizTalk Server tips and tricks for developers and admins” presented in BizTalk Summit 2015 London is now available online

My session about “BizTalk Server tips and tricks for developers and admins” presented in BizTalk Summit 2015 London is now available online

For the third year BizTalk Summit 2015 London was I huge success, for that we need to thanks BizTalk360 team and Saravana Kumar for being able to assemble this great event. My session This year I only had 30 minutes to speak, so I had to improvise and try to create a lightweight, funny but […]
Blog Post by: Sandro Pereira

Thoughts on Integration Architecture and Analysis.

Thoughts on Integration Architecture and Analysis.

Apologies for not being a regular blogger these days, this is my first post from Australia. I have joined new company and moved to Brisbane – a beautiful Queensland city.

This post is mainly related to the Integration competency where I am just going to share good work being done by the architect(s) to make feel better and speed up the new comer(s) in the company like myself. I have been lacking with the terms in the previous engagement(s) like Business Service, Business Events, Integration points, Product Adapters, Channel Adapters Etc…Etc… These are key components of an Integration projects and need to be well define in the analysis phase. A proper maintenance in Wiki shall be ideal for people to understand the Integration solution.

The steps which shall be followed in the Architecture/Analysis phase are:

  • Define Business Services/Business Events. For eg: If the message needs to be send from Party A to Party B to create purchase order. A Business service needs to be created like “PurchaseOrderManagement” and Business Events need to be define as CreatePurchaseOrder(), UpdatePurchaseOrder(), RetreivePurchaseOrder etc.

Creating Wiki page or register where you can add the top level “Business Services” and sub levels Business Events               (or operations) shall be ideal. Give definition to your Business services and include all the business events.

Screen for the business service register should look like below:

The name of the service register should be “PurchaseOrderManagement”. The service operations are the different              events which will act as a business service. Like CreatePurchaseorder, UpdatePurchaseOrder,                                RetreivePurchaseOrder etc.

  • Each business service should have its own page to describe and define message exchange patterns 1-way, 2-way request-response etc, Integration Points, Sequence diagram, data contract, Faults (business/system), request message schema and sample, response message schema and sample, end points etc.

The page should look like this:

  • Integration point diagram (component diagram) and sequence diagram should be linked or added to the service operation page.
  • Integration Point Diagram: – This is a component diagram reflecting the integration points. These integration points are in the integration layer and needs to be developed in ESB (WCF/BizTalk).

The above diagram shows the front end caller, Integration Layer with the channel adapter and the ESB business                   service and end systems. On the same page there is a technical design section to provide the links to the business               service operations this integration point is link to.

The architecture and analysis phase are more important in the life cycle of the integration project, there are many chances that things can go wrong if proper thoughts and consideration is not taken into account, otherwise we might end up doing re-factoring or re-designing the solution.

Thanks.

Advertisements

Skip Create BizTalk Application using BTDF

Skip Create BizTalk Application using BTDF

This article is to demonstrate how to skip create BizTalk Application using BTDF. By default the btdf project will call the target “DeployAppDefinition”. This target is responsible to create BizTalk Application when it is not exists in the Admin Console, otherwise it throw an error when the application is found.

In my current project we were require to only deploy the version assemblies for the particular application without removing the BT application.

I came up with the below BTDF Project which is using the custom target to call DeployAppDefinition which checks for the existence of the application and do not remove if exists.

I need to create this variable in the property group “<SkipRemoveApp>True</ SkipRemoveApp >  “ and used it in the DeployAppDefinition target.

<!– Skip BizTalk application definition –>

<Target Name=”DeployAppDefinition” Condition=”‘$( SkipRemoveApp)’ == ‘False’”>

<Exec Command=”BTSTask.exe AddApp -ApplicationName:&quot;$(BizTalkAppName)&quot; -Description:&quot;$(ProjectName)&quot;” />

<AddAppReference ApplicationName=”$(BizTalkAppName)” AppsToReference=”@(AppsToReference)” Condition=”%(Identity) == %(Identity) and ‘@(AppsToReference)’ != ”” />

</Target>

Below is the complete btdf project. Please let me know if anyone having a different approach.

<?xml version=”1.0″ encoding=”utf-8″?>

<!–

***************************************************

**** DEPLOYMENT STEPS *****************************************************************************************************

This deployment package will only add BizTalk resources to the existing application.

***************************************************************************************************************************

–>

<Project xmlns=”http://schemas.microsoft.com/developer/msbuild/2003&#8243;

DefaultTargets=”Installer”

ToolsVersion=”4.0″

TreatAsLocalProperty=”SkipUndeploy”> <!– **** Add this so this variable is accessible on this target file ****–>

<PropertyGroup>

<Configuration Condition=”‘$(Configuration)’ == ””>Debug</Configuration>

<Platform Condition=”‘$(Platform)’ == ””>x86</Platform>

<SchemaVersion>1.0</SchemaVersion>

<ProjectName>ESB.ReferenceSystems.P</ProjectName>

<ProjectVersion Condition=”‘$(ProjectVersion)’ == ””>1.1</ProjectVersion>

<ProjectVersion Condition=”‘$(ProjectVersion)’ != ””>$(ProjectVersion)</ProjectVersion>

<IncludeSchemas>False</IncludeSchemas>

<IncludeOrchestrations>False</IncludeOrchestrations>

<IncludeTransforms>False</IncludeTransforms>

<IncludePipelines>False</IncludePipelines>

<IncludeComponents>False</IncludeComponents>

<IncludePipelineComponents>False</IncludePipelineComponents>

<IncludeCustomFunctoids>False</IncludeCustomFunctoids>

<IncludeVocabAndRules>False</IncludeVocabAndRules>

<IncludeVirtualDirectories>False</IncludeVirtualDirectories>

<IncludeMessagingBindings>False</IncludeMessagingBindings>

<IncludeDeploymentTest>False</IncludeDeploymentTest>

<Includelog4net>False</Includelog4net>

<IncludeSSO>False</IncludeSSO>

<IncludeEsbItineraries>False</IncludeEsbItineraries>

<IncludeBam>False</IncludeBam>

<IncludeInstallUtilForComponents>False</IncludeInstallUtilForComponents>

<UsingMasterBindings>True</UsingMasterBindings>

<RequireXmlPreprocessDirectives>False</RequireXmlPreprocessDirectives>

<ApplyXmlEscape>True</ApplyXmlEscape>

<IncludeSettingsSpreadsheetInMsi>True</IncludeSettingsSpreadsheetInMsi>

<SkipIISReset>True</SkipIISReset>

<SkipHostInstancesRestart>False</SkipHostInstancesRestart>

<StartApplicationOnDeploy>True</StartApplicationOnDeploy>

<EnableAllReceiveLocationsOnDeploy>False</EnableAllReceiveLocationsOnDeploy>

<StartReferencedApplicationsOnDeploy>False</StartReferencedApplicationsOnDeploy>

<UseIsolatedAppDomain>False</UseIsolatedAppDomain>

<EnableBizTalkExtendedLogging>False</EnableBizTalkExtendedLogging>

<EnableBizTalkAssemblyValidation>False</EnableBizTalkAssemblyValidation>

<EnableBizTalkCorrelationValidation>False</EnableBizTalkCorrelationValidation>

<EnableBizTalkSchemaValidation>False</EnableBizTalkSchemaValidation>

<SkipAddApp>True</SkipAddApp>

</PropertyGroup>

<PropertyGroup>

<!– Properties related to building an MSI for server deployments –>

<!– BizTalk App Version Upgrade –>

<!–   For each new product release to be deployed to your BizTalk servers: –>

<!–     1) Increment ProductVersion –>

<!–     2) Generate a new GUID and update ProductId with the new GUID –>

<!–   This allows the new MSI to automatically uninstall (not undeploy!) the old MSI and install the new one. –>

<ProductVersion Condition=”‘$(ProductVersion)’ == ””>1.0.1</ProductVersion>

<ProductVersion Condition=”‘$(ProductVersion)’ != ””>$(ProductVersion)</ProductVersion>

<ProductId Condition=”‘$(ProductId)’ == ””>A66F460E-7B65-4330-843B-477251BF3AEC</ProductId>

<ProductId Condition=”‘$(ProductId)’ != ””>$(ProductId)</ProductId>

<!– BizTalk App Version Upgrade –>

<ProductName>ESB.ReferenceSystems.P for BizTalk</ProductName>

<Manufacturer></Manufacturer>

<PackageDescription>ESB.ReferenceSystems.P</PackageDescription>

<PackageComments>ESB.ReferenceSystems.P</PackageComments>

<!– NEVER change the ProductUpgradeCode. –>

<ProductUpgradeCode>A87DEA9B-53B5-4A30-85E3-71F5C698D203</ProductUpgradeCode>

<IsInTFSBuildMode Condition=”Exists(‘$(OutDir)$(Configuration)ESB.ReferenceSystems.P.SubmissionProcessAPI.Schemas.dll’)”>True</IsInTFSBuildMode>

</PropertyGroup>

<!– Under TFS Team Build, set CustomizableOutDir property to true in TFS 2005/2008/2010 UpgradeTemplate. –>

<!– With a workflow build, copy the default template then modify the MSBuild task for the solution build. Set OutDir to blank and –>

<!– CommandLineArguments to String.Format(“/p:SkipInvalidConfigurations=true;TeamBuildOutDir=””{0}”” {1}”, BinariesDirectory, MSBuildArguments). –>

<PropertyGroup Condition=”‘$(Configuration)’ == ‘Debug’”>

<DeploymentFrameworkTargetsPath>$(MSBuildExtensionsPath)DeploymentFrameworkForBizTalk5.0</DeploymentFrameworkTargetsPath>

<OutputPath Condition=”‘$(TeamBuildOutDir)’ == ””>binDebug</OutputPath>

<OutputPath Condition=”‘$(TeamBuildOutDir)’ != ””>$(TeamBuildOutDir)</OutputPath>

<DeployPDBsToGac>false</DeployPDBsToGac>

</PropertyGroup>

<PropertyGroup Condition=”‘$(Configuration)’ == ‘Release’”>

<DeploymentFrameworkTargetsPath>$(MSBuildExtensionsPath)DeploymentFrameworkForBizTalk5.0</DeploymentFrameworkTargetsPath>

<OutputPath Condition=”‘$(TeamBuildOutDir)’ == ””>binRelease</OutputPath>

<OutputPath Condition=”‘$(TeamBuildOutDir)’ != ””>$(TeamBuildOutDir)</OutputPath>

<DeployPDBsToGac>false</DeployPDBsToGac>

</PropertyGroup>

<PropertyGroup Condition=”‘$(Configuration)’ == ‘Server’”>

<DeploymentFrameworkTargetsPath>Framework</DeploymentFrameworkTargetsPath>

<!– Get our PDBs into the GAC so we get file/line number information in stack traces. –>

<DeployPDBsToGac>true</DeployPDBsToGac>

</PropertyGroup>

<ItemGroup>

<PropsFromEnvSettings Include=”SsoAppUserGroup;SsoAppAdminGroup” />

</ItemGroup>

<ItemGroup>

<AdditionalFiles Include=”Deployment.config”>

<LocationPath>.</LocationPath>

</AdditionalFiles>

</ItemGroup>

<ItemGroup>

<!–<AppsToRemove Include=”ESB.ReferenceSystems.P” />–>

</ItemGroup>

<!– !!! TODO !!! –>

<Import Project=”$(DeploymentFrameworkTargetsPath)BizTalkDeploymentFramework.targets” />

<!–

The Deployment Framework automatically packages most files into the server install MSI.

However, if there are special folders that you need to include in the MSI, you can

copy them to the folder $(RedistDir) in the CustomRedist target.

To include individual files, add an ItemGroup with AdditionalFiles elements.

–>

<!–

The Targets below are to enable undeploy during the TFS build when not deployed – Bill Chesnut

–>

<!– **** PRE DEPLOYMENT SECTION ******************************************************************************************–>

<!– **********************************************************************************************************************–>

<!– **** The BTDF out of the box calls this target before it deploys the application                                  ****–>

<!– **** Set the “SkipUndeploy” to false so that we can undeploy the application before deployment                    ****–>

<!– **** Then call the Custom Deploy Target to add the undeployment step                                              ****–>

<!– **********************************************************************************************************************–>

<Target Name=”CustomDeployTarget”>

<PropertyGroup>

<SkipUndeploy>false</SkipUndeploy>

</PropertyGroup>

<CallTarget Targets=”CustomDeploy” />

</Target>

<!– **********************************************************************************************************************–>

<!– **** This target is called from CustomDeployTarget                                                                ****–>

<!– **** DeployAppDefinition is an out of the box target  which will skip the check for Add Application               ****–>

<!– **** CustomUndeployApp is the custom target created to undeploy the application                                   ****–>

<!– **** CustomDeployAssemblyTarget is the custom target to Add the BizTalk Assemblies                                ****–>

<!– **********************************************************************************************************************–>

<Target Name=”CustomDeploy” DependsOnTargets=”$(CustomDeployTargetDependsOn)” />

<PropertyGroup>

<!– CustomDeploy depends on this CustomDeployTargetDependsOn –>

<CustomDeployTargetDependsOn>

DeployAppDefinition;

CustomDeployAssemblyTarget;

</CustomDeployTargetDependsOn>

</PropertyGroup>

<!– Skip BizTalk application definition –>

<Target Name=”DeployAppDefinition” Condition=”‘$(SkipAddApp)’ == ‘False’”>

<Exec Command=”BTSTask.exe AddApp -ApplicationName:&quot;$(BizTalkAppName)&quot; -Description:&quot;$(ProjectName)&quot;” />

<AddAppReference ApplicationName=”$(BizTalkAppName)” AppsToReference=”@(AppsToReference)” Condition=”%(Identity) == %(Identity) and ‘@(AppsToReference)’ != ”” />

</Target>

<Target Name=”CustomDeployAssemblyTarget” DependsOnTargets=”$(CustomDeployAssemblyTargetDependsOn)” />

<PropertyGroup>

<!– CustomPostDeployTarget depends on this CustomPostDeployTargetDependsOn –>

<CustomDeployAssemblyTargetDependsOn>

AddBizTalkResources;

</CustomDeployAssemblyTargetDependsOn>

</PropertyGroup>

<!– **********************************************************************************************************************–>

<!– **** This target is responsible for adding all the biztalk resources into the application resources and is called ****–>

<!– **** from CustomDeployAssemblyTarget                                                                                     ****–>

<!– **********************************************************************************************************************–>

<Target Name=”AddBizTalkResources” DependsOnTargets=”$(AddBizTalkResourcesDependsOn)” />

<PropertyGroup>

<!– AddBizTalkResources depends on this AddBizTalkResourcesDependsOn –>

<AddBizTalkResourcesDependsOn>

AddBizTalkAssembliesToBizTalkResources;

</AddBizTalkResourcesDependsOn>

</PropertyGroup>

<!– **********************************************************************************************************************–>

<!– **** This target is responsible for adding the BizTalk assemblies into the application resources and is called       ****–>

<!– **** from AddBizTalkResources                                                                                     ****–>

<!– **********************************************************************************************************************–>

<Target Name=”AddBizTalkAssembliesToBizTalkResources”>

<Message Text=”In AddBizTalkAssembliesToBizTalkResources.-2″/>

<Exec

Command=”BTSTask.exe AddResource -Type:BizTalkAssembly  -Source:&quot;..SubmissionProcessAPI.SchemasbinDebugESB.ReferenceSystems.P.SubmissionProcessAPI.Schemas.dll&quot; -ApplicationName:&quot;$(BizTalkAppName)&quot; -Overwrite -Options:GacOnAdd,GacOnImport,GacOnInstall”

Condition=”‘$(DeployBizTalkMgmtDB)’ == ‘true’ ” />

</Target>

</Project>

Advertisements

WCF-SAP Adapter: Finding the document specification by message type [IDOC Message Type] failed. Verify the schema deployed properly.

WCF-SAP Adapter: Finding the document specification by message type [IDOC Message Type] failed. Verify the schema deployed properly.

In the sequence of my last post, and following the result obtained while trying to solve the problems that had been originated by the SAP system migration to a newer version, I end up receiving another error. This time the connectivity problem with SAP was overcome, however without any reason I started to receive an […]
Blog Post by: Sandro Pereira

BizTalk WCF-SAP Adapter: RFC IDOC_INBOUND_ASYNCHRONOUS could not be resolved against SAP system because its metadata could not be obtained

BizTalk WCF-SAP Adapter: RFC IDOC_INBOUND_ASYNCHRONOUS could not be resolved against SAP system because its metadata could not be obtained

Recently one of my clients migrated one of their SAP systems and after that intervention the BizTalk receive location that was listening in a specific Program ID started to failed with the following error message present in the event viewer: “RFC IDOC_INBOUND_ASYNCHRONOUS could not be resolved against SAP system because its metadata could not be […]
Blog Post by: Sandro Pereira