Parsing plain text messages/flat files received through POP3 adapter

by community-syndication | Mar 31, 2006 | BizTalk Community Blogs via Syndication

Recently I received this question from one of our customers, unfortunately his email address was wrong and I couldn’t reply directly to him. Before leaving to the BizTalk Shipping party … yuuhuuu… I thought to try and provide an answer. See below a scrubbed version of the email I received

I am trying to find someone to help me with few tips related to BizTalk 2006 and POP3 adapter – specifically parsing plain text messages as they are flat files.

We are currently evaluating BizTalk 2006. Parsing of the plain text email messages containing for example orders as the body of the message is one of our tasks.

Do you know of any good resource (blog, samples, …) to recommend.

I already tried community newsgroups but no help there. I would expect that this is rather trivial task with BizTalk 2006.

POP3 adapter has a built-in MIME decoding feature which is enabled by default (“Apply MIME Decoding” property on POP3 receive location property page). So, when POP3 adapter receives a message, it will convert the email body into BizTalk message part. It will save other email headers on the BizTalk message context. You can use Flat-file disassembler component in the receive pipeline and retrieve the actual business data. You can find some basic information on POP3 adapter here: http://www.microsoft.com/biztalk/techinfo/whitepapers/adapterwp.mspx (Open BTS2k6AdapterEnhance.doc)

Here is the online documentation for BizTalk Server 2006.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bts06default/html/cadd3bdd-f13b-427d-8979-b2541aa5e21d.asp

If there are other questions, please follow up with me and I’ll try to get some answers back to you. Please make sure you fill-in your email address correctly.

Parsing plain text messages/flat files received through POP3 adapter

by community-syndication | Mar 31, 2006 | BizTalk Community Blogs via Syndication

Recently I received this question from one of our customers, unfortunately his email address was wrong and I couldn’t reply directly to him. Before leaving to the BizTalk Shipping party … yuuhuuu… I thought to try and provide an answer. See below a scrubbed version of the email I received

I am trying to find someone to help me with few tips related to BizTalk 2006 and POP3 adapter – specifically parsing plain text messages as they are flat files.

We are currently evaluating BizTalk 2006. Parsing of the plain text email messages containing for example orders as the body of the message is one of our tasks.

Do you know of any good resource (blog, samples, …) to recommend.

I already tried community newsgroups but no help there. I would expect that this is rather trivial task with BizTalk 2006.

POP3 adapter has a built-in MIME decoding feature which is enabled by default (“Apply MIME Decoding” property on POP3 receive location property page). So, when POP3 adapter receives a message, it will convert the email body into BizTalk message part. It will save other email headers on the BizTalk message context. You can use Flat-file disassembler component in the receive pipeline and retrieve the actual business data. You can find some basic information on POP3 adapter here: http://www.microsoft.com/biztalk/techinfo/whitepapers/adapterwp.mspx (Open BTS2k6AdapterEnhance.doc)

Here is the online documentation for BizTalk Server 2006.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bts06default/html/cadd3bdd-f13b-427d-8979-b2541aa5e21d.asp

If there are other questions, please follow up with me and I’ll try to get some answers back to you. Please make sure you fill-in your email address correctly.

XML Reusability

by community-syndication | Mar 30, 2006 | BizTalk Community Blogs via Syndication

Here are the instructions on how to create xml objects and then reuse them in a different schema.

1. Create your object that you want to reuse in a schema
2. In the properties of the object, type in objectType in the Data Structure Type
3. Create your ’real’ schema and in the Schema node’s properties, click the elipse on Imports and browse to the schema defined in Step 1
4. Create the real object and in the Data Structure Type, choose the complex type that was created in Step 2

Customizing BizTalk Web Services deployment

by community-syndication | Mar 30, 2006 | BizTalk Community Blogs via Syndication

Out of the box, basic web project deployment is taken care of as soon as web port is configured in BizTalk application and MSI is exported. When running, this MSI will create web directory under default IIS web site and copy all files found in this directory on the build server. Now, what if we want to change physical location of web directory to be different from the build server, or limit set of files that must be deployed, or deploy precompiled web project (by source files get deployed)? There are ways to achieve these goals.

To add web directory to an application when building a package we can use following command:

BtsTask.exe AddResource -Type:System.BizTalk:WebDirectory

 /Source:value /Destination:value

There’s one problem though: the Destination parameter defines only virtual path. The physical path will be the same as on the source machine where the package is built. We can change it by creating IIsVirtualDir metabase entry with Path property set to desired physical location prior to calling AddResource command. This will still leave us with physical path hardwired into the msi without ability to alter it at the install time. Another problem with this approach is that all files from source directory will be added to the package as is (not compiled source files) which is not always desired.

More flexible way would be to take full control of the web project installation process and here is how.

Package build:

1)       Compile web project in desired mode to designated folder (in this example PrecompiledPath) using aspnet_compiler.exe. There are three compilation modes available for web projects: default no precompilation (source code, will be compiled on request), precompiled updateable, binaries only. Pasting following code into project’s build subscript (i.e. %AppName%_build_projects.bat in the previous post) will take care of this step:

IF %BuildMode%==Debug (SET WebEmitDebug=-d)

aspnet_compiler -v /%VirtDirName% -f -fixedNames %WebEmitDebug% “PrecompiledPath“

2)       Add required only files from this folder to the package using BtsTask AddResource. One way of doing this:

FOR %%F IN (“PrecompiledPath\*.*”) DO btstask AddResource /A:%AppName% /Type:System.BizTalk:File /Source:”%%F” /Destination:”%%BTAD_InstallDir%%\TargetFolderName\%%~xnF” /Overwrite

FOR %%F IN (“PrecompiledPath\bin\*.*”) DO btstask AddResource /A:%AppName% /Type:System.BizTalk:File /Source:”%%F” /Destination:”%%BTAD_InstallDir%%\TargetFolderName\bin\%%~xnF” /Overwrite

3)       Create script that will configure virtual directory during installation when invoked. The script can use whether WMI or ADSI. Here’s an example of WMI one:

Sub ConfigureWebApplication(installPath, appPoolName, webDirName, serviceAccount, password)

    Dim iisvdirCmd, winDir, objWMI, items, item

    Const OutOfProcess = 1

    Const WaitOnReturn = True

    Const CreateIfNotExist = True

    Const WindowStyle = 0

    Set shell = CreateObject(“WScript.Shell”)

    Set environment = shell.Environment(“Process”)

    winDir = environment(“WINDIR”)

    ‘Create virtual directory

    iisvdirCmd = winDir & “\system32\iisvdir.vbs /create “ & Chr(34) & “Default Web Site” & Chr(34) & ” “ & webDirName & ” “ & _

     Chr(34) & installPath & “\” & webDirName & Chr(34)

    shell.Run iisvdirCmd, WindowStyle, WaitOnReturn

    Set objWMI = GetObject (“winmgmts:{authenticationLevel=pktPrivacy}\\.\root\microsoftiisv2”)

    Set items = objWMI.ExecQuery(“Select * From IIsWebVirtualDir Where Name = ‘W3SVC/1/ROOT/” & webDirName & “‘”)

    For Each item in items

       item.AppCreate3 OutOfProcess, appPoolName, CreateIfNotExist

       Wscript.Echo “Creating web application “ & item.Name

    Next

    Set items = objWMI.ExecQuery(“Select * From IIsWebVirtualDirSetting Where Name = ‘W3SVC/1/ROOT/” & webDirName & “‘”)

    For Each item in items

        Wscript.Echo “Configuring security for virtual directory “ & item.Name         

        item.AccessSSL = True

        item.AccessSSL128 = True

        item.AccessSSLRequireCert = True

        item.AppFriendlyName = webDirName

        item.Put_    

    Next

    ‘Set application pool identity

    Set items = objWMI.ExecQuery(“Select * from IIsApplicationPoolSetting Where Name = ‘W3SVC/apppools/” & appPoolName & “‘”)

    For Each item in items

        Wscript.Echo “Setting identity of application pool “ & item.Name

        item.WAMUserName = serviceAccount

        item.WAMUserPass = password

        item.Put_

    Next

End Sub

This function not only creates virtual directory but completely configures web application to be used by BizTalk including setting up application pool and its identity. There are also functions to create folders, write to registry and do other useful tasks that I don’t show here.

We have to make sure our script is invoked in right context, so it can be like this:

          CreateFolders(path)

          WriteToRegistry(path)

          ConfigureWebApplication(path, appPoolName, webDirName, serviceAccount, password)

      End If

End Sub

Finally, let’s add this script to the package as preprocessing (install stage):

btstask AddResource /A:%AppName% /Type:System.BizTalk:PreProcessingScript

 /Source:PreInstallScript.vbs /Overwrite

I am not working for Microsoft Japan, but thanks a lot Jeff!

by community-syndication | Mar 30, 2006 | BizTalk Community Blogs via Syndication

In a recent entry , Jeff noticed my Debugger Visualizer for MessageContext and mentioned that I am from MSFT Japan. Well, this is not true. I am in Redmond, USA and I work for the BizTalk team. It it true that I do like Japan and I have visited this gret…(read more)

BizTalk 2006 Scenarios

by community-syndication | Mar 29, 2006 | BizTalk Community Blogs via Syndication

BizTalk Server 2006 is now release and available out on MSDN.  I’ve loaded up an image and was curious to see if I could get one of the new Scenario Examples working.  If you haven’t seen the new scenarios, install BizTalk and look at the following location:  c:\Program Files\BizTalk Server 2006\SDK\Scenarios\  I started looking at the BPM Sample and it’s not just a code example, but a full working application.

If you want to install this scenario, I suggest you first look at the Help File!  The new help file has a scenario link on the initial page and should lead you to the instructions.  I printed off the BPM instructions and it’s not just a click and go.  It’s quite a few pages with with everything from setting up the DB, Queues, FTP, etc…  Don’t expect to get this scenario running in 5 minutes.  One interesting not on the instructions…it states that you should place the BizTalk Service Account in the Administrators group…so much for using least priviledge!

I haven’t quite figured out the entire BPM solution and it will take some time.  To get the app running, you must start three separate applications:  Facilities, Operations and Provisioning.  These apps simulate the back end systems so you can run the application on one machine.  I was able to get everything working and started submitting service requests.  The only issue I’ve run into so far has been BAM.  There is BAM spread throughout the Solution, but nothing displayed in the BAM Portal.  I had to run the bm.exe tool and add an account to the ServiceOrderView.  Once I did this, I could see the BAM view and start running queries.

It’s great to see a full solution sample in the BizTalk SDK.  It will take some time to dig into this Solution and figure out all of the moving parts, but it should help answer quite a few questions.  I do some more digging also and if I find anything interesting, I’ll let everyone know.

Upgrading from BT’04 GotDotNet WSS Adapter 1.0 to BT’06 native WSS Adapter

by community-syndication | Mar 28, 2006 | BizTalk Community Blogs via Syndication

I finally found some time and tried upgrading from BizTalk 2004 and GotDotNet WSS adapter to BizTalk 2006 and the native BT’06 adapter for Windows SharePoint Services. The idea was to start from a BizTalk 2004 installation with version 1.0 of Steve Resnick & Co. WSS Adapter (http://www.gotdotnet.com/workspaces/workspace.aspx?id=0d1aa85c-cf8d-497e-84f4-3ffec8db115f) , setup a simple messaging scenario using SharePoint, upgrade to BizTalk 2006, verify that messaging still works, upgrade to the native BizTalk 2006 WSS Adapter and then verify that messaging works with the new adapter.

I followed the steps below, in order to create a sample messaging test for SharePoint:

1. Created Source document library, then renamed it SourceDocLib (this trick causes the document library URL and the document library name to be different and this way I can see if WSSLib adapter really expects the URL or the display name of the document library). Created a View named ’UpgradeDocuments’ that shows only those documents with ID >= 0 (probably all docs).
   
   
2. Created document library named ‘Destination’, then renamed it to ‘DestinationDocLib’. Created folder ’Dest Folder’. Added columns SharePointColumn1, and SharePointColumn2.
   
   
3. Created document library named ‘Archive’, then renamed it to ‘ArchiveDocLib’. Created ’My Folder’ folder in the library just created.
   
   
4. Created receive port and receive location for WSSLib adapter. Used all the fields available in order to make sure that similar field exists in native BT’06 adapter.
   
   
5. Created send port for WSSLib adapter, used all the fields.
   (See the exported bindings or the UI captures below to see how the ports were configured. If you want to repro my test, after importing the bindings, you will have to update the filter on the send port to process messages received from the receive location by adding BTS.ReceivePortName == WSSLib-Receive condition. Only then start the send port.)
   
   
6. Sent test messages (see attached).
   
   
7. Exported all bindings.

See images below for the receive location and send port settings:

Once I configured the messaging I’ve sent a couple of messages through (see attached) to make sure that everything is working fine.

UPGRADE

I followed these steps http://geekswithblogs.net/darko/archive/2006/01/24/66866.aspx in order to upgrade to BizTalk 2006. The upgrade went through without problems. Another place where you can find upgrade information is this blog http://blogs.msdn.com/biztalk_upgrade/default.aspx Because my BizTalk 2004 installation had the BAS feature installed, after upgrade, both BAS and Windows SharePoint Services Adapter Web Service features were installed on the machine (even though the latter does not show up as enabled in setup, it’s there because BAS requires it). If your BizTalk 2004 installation didn’t have BAS, after upgrading your SharePoint box, you will have to start setup, add Windows SharePoint Services Adapter Web Service feature, and configure it. That’s easy and straight forward so I won’t go into details.

After upgrade, I browsed to the BAS site (SharePoint site) to see if SharePoint was still working fine and it didn’t. I received the following error:
This Windows SharePoint Services virtual server has not been configured for use with ASP.NET 2.0.50727.42. For more information, please refer to Knowledge Base article 894903 at http://go.microsoft.com/fwlink/?LinkId=42660.

Troubleshoot issues with Windows SharePoint Services

The KB article just tells you to run one command. I run the command below:

stsadm.exe -o upgrade -forceupgrade -url http://demoappserver/sites/BASSite

restarted IIS and after that the BAS site shown up without problems. I started the BizTalk services that I stopped during upgrade, sent a few SharePoint messages through and everything worked fine.

Updating the ports

So I proceeded to update the receive location and send port in order to change from WSSLib adapter to ‘Windows SharePoint Services’ adapter. First thing I did, was to stop the BizTalk Host Instances that were used by the WSSLib adapter. This is probably not necessary, but in a couple of cases I noticed that the WSSLib ports don’t shutdown appropriately and I didn’t want to run into issues. After that, for each of the receive location or send port I changed the ‘Transport Type’ property from ‘WSSLib’ to ‘Windows SharePoint Services’ and then re-configured the send port/receive location like below.

Once I updated all the send ports and receive locations, I re-started the host instances, and sent a few messages through. All the messages were processed correctly in the same manner as before and this concluded my upgrade experience. Please notice that my upgrade test was done using version 1.0 of the WSSLib adapter. On GotDotNet there is an RC for the WSSLib adapter version 2.0. A comparision of that WSSLib2 adapter and BizTalk 2006 native ‘Windows SharePoint Services’ adapter can be found here http://blogs.msdn.com/ahamza/archive/2005/07/27/BizTalk2006WSSAdapters.aspx The only feature supported by WSSLib2 that is not available in the native BT’06 adapter is storing the Username/Password in SSO per send port/receive location. The native BT’06 adapter uses the identity of the host instance to connect and authenticate with SharePoint. Upgrade from WSSLib2 to BT’06 native WSS Adapter should be just as straight forward.

Regardless of the adapter version from which you are upgrading, if you are using some of the adapter context properties in your orchestrations you will have to update your orchestrations to use the WSS.* context properties and that will require you to recompile and redeploy your orchestrations.

Upgrading from BT’04 GotDotNet WSS Adapter 1.0 to BT’06 native WSS Adapter

by community-syndication | Mar 28, 2006 | BizTalk Community Blogs via Syndication

I finally found some time and tried upgrading from BizTalk 2004 and GotDotNet WSS adapter to BizTalk 2006 and the native BT’06 adapter for Windows SharePoint Services. The idea was to start from a BizTalk 2004 installation with version 1.0 of Steve Resnick & Co. WSS Adapter (http://www.gotdotnet.com/workspaces/workspace.aspx?id=0d1aa85c-cf8d-497e-84f4-3ffec8db115f) , setup a simple messaging scenario using SharePoint, upgrade to BizTalk 2006, verify that messaging still works, upgrade to the native BizTalk 2006 WSS Adapter and then verify that messaging works with the new adapter.

I followed the steps below, in order to create a sample messaging test for SharePoint:

1. Created Source document library, then renamed it SourceDocLib (this trick causes the document library URL and the document library name to be different and this way I can see if WSSLib adapter really expects the URL or the display name of the document library). Created a View named ’UpgradeDocuments’ that shows only those documents with ID >= 0 (probably all docs).
   
   
2. Created document library named ‘Destination’, then renamed it to ‘DestinationDocLib’. Created folder ’Dest Folder’. Added columns SharePointColumn1, and SharePointColumn2.
   
   
3. Created document library named ‘Archive’, then renamed it to ‘ArchiveDocLib’. Created ’My Folder’ folder in the library just created.
   
   
4. Created receive port and receive location for WSSLib adapter. Used all the fields available in order to make sure that similar field exists in native BT’06 adapter.
   
   
5. Created send port for WSSLib adapter, used all the fields.
   (See the exported bindings or the UI captures below to see how the ports were configured. If you want to repro my test, after importing the bindings, you will have to update the filter on the send port to process messages received from the receive location by adding BTS.ReceivePortName == WSSLib-Receive condition. Only then start the send port.)
   
   
6. Sent test messages (see attached).
   
   
7. Exported all bindings.

See images below for the receive location and send port settings:

Once I configured the messaging I’ve sent a couple of messages through (see attached) to make sure that everything is working fine.

UPGRADE

I followed these steps http://geekswithblogs.net/darko/archive/2006/01/24/66866.aspx in order to upgrade to BizTalk 2006. The upgrade went through without problems. Another place where you can find upgrade information is this blog http://blogs.msdn.com/biztalk_upgrade/default.aspx Because my BizTalk 2004 installation had the BAS feature installed, after upgrade, both BAS and Windows SharePoint Services Adapter Web Service features were installed on the machine (even though the latter does not show up as enabled in setup, it’s there because BAS requires it). If your BizTalk 2004 installation didn’t have BAS, after upgrading your SharePoint box, you will have to start setup, add Windows SharePoint Services Adapter Web Service feature, and configure it. That’s easy and straight forward so I won’t go into details.

After upgrade, I browsed to the BAS site (SharePoint site) to see if SharePoint was still working fine and it didn’t. I received the following error:
This Windows SharePoint Services virtual server has not been configured for use with ASP.NET 2.0.50727.42. For more information, please refer to Knowledge Base article 894903 at http://go.microsoft.com/fwlink/?LinkId=42660.

Troubleshoot issues with Windows SharePoint Services

The KB article just tells you to run one command. I run the command below:

stsadm.exe -o upgrade -forceupgrade -url http://demoappserver/sites/BASSite

restarted IIS and after that the BAS site shown up without problems. I started the BizTalk services that I stopped during upgrade, sent a few SharePoint messages through and everything worked fine.

Updating the ports

So I proceeded to update the receive location and send port in order to change from WSSLib adapter to ‘Windows SharePoint Services’ adapter. First thing I did, was to stop the BizTalk Host Instances that were used by the WSSLib adapter. This is probably not necessary, but in a couple of cases I noticed that the WSSLib ports don’t shutdown appropriately and I didn’t want to run into issues. After that, for each of the receive location or send port I changed the ‘Transport Type’ property from ‘WSSLib’ to ‘Windows SharePoint Services’ and then re-configured the send port/receive location like below.

Once I updated all the send ports and receive locations, I re-started the host instances, and sent a few messages through. All the messages were processed correctly in the same manner as before and this concluded my upgrade experience. Please notice that my upgrade test was done using version 1.0 of the WSSLib adapter. On GotDotNet there is an RC for the WSSLib adapter version 2.0. A comparision of that WSSLib2 adapter and BizTalk 2006 native ‘Windows SharePoint Services’ adapter can be found here http://blogs.msdn.com/ahamza/archive/2005/07/27/BizTalk2006WSSAdapters.aspx The only feature supported by WSSLib2 that is not available in the native BT’06 adapter is storing the Username/Password in SSO per send port/receive location. The native BT’06 adapter uses the identity of the host instance to connect and authenticate with SharePoint. Upgrade from WSSLib2 to BT’06 native WSS Adapter should be just as straight forward.

Regardless of the adapter version from which you are upgrading, if you are using some of the adapter context properties in your orchestrations you will have to update your orchestrations to use the WSS.* context properties and that will require you to recompile and redeploy your orchestrations.

BizTalk Server 2006 Installation Guides and Tutorials

by community-syndication | Mar 28, 2006 | BizTalk Community Blogs via Syndication

BizTalk Server 2006 RTM Now Available on MSDN

by community-syndication | Mar 28, 2006 | BizTalk Community Blogs via Syndication

Very good news !!!

BizTalk Server 2006 is now available for download on the MSDN Subscribers Download site!

Also available are the SWIFT, RosettaNet, HL7, and HIPPA Accelerators as well as the Enterprise Application Adapters.