Another BizTalk Presentation

I always forget to post these in advance.


I am doing another BizTalk Presentation at Btug, tomorrow night (March 28) in Toronto.


It is split into two parts:
i)Handling Large Messages in BizTalk
ii)Extending BizTalk Transformations into Sql Server


Also ObjectSharp will be giving away a free seat for a 5 day course -> Building BizTalk 2006 Solutions Best Practices with the first delivery of the course slated for May 1 2006. You can read more about the presentation Here

One more EventStream!

A quick correction on my earlier BAM posting, there are actually four EventStreams in BizTalk Server 2006, there is the MessagingEventStream which operates in the same way as the OrchestrationEventStream in that it’s transactionally consistent with the pipeline execution “transaction” and should be used within a custom pipeline.


Thanks to Keith Lim for spotting my mistake, who by the way has a great BAM focused blog which is great to see!

Secure Shell (SSH) Enabled FTP Adapter for BizTalk


Since there is not enough information available on the sFTP adapters for
BizTalk, thought would post this one, you can find the article here,
sFTP adapters for BizTalk

This document is more focused on the BizTalk side rather than SSH, however I
have provided some basic background information on
SSH servers and clients using the RSA key based authentication in my previous post.




BizTalk 2004/2006 doesn’t have any built in support for sFTP adapters, they
only support traditional FTP adapters, hence we are only left with two options,
either we need to develop our own custom adapters for sFTP or purchase it from
other clients, for more information read the article I had posted on the
sFTP adapter.

RSA Public-Key based Authentication

 


BizTalk Tidbits

BizTalk Tidbits

I have a bunch of tidbits that I have found, and instead of creating a blog entryfor each of them, I have put them alltogether into one big entry.I hope that there issomething in this foreveryone!

Items Explained:
1.Using the Flat File Schema you can define custom date formats. One word of caution, in order to use this feature, you have to guarantee that the date is going to be present.

2.Using XPath, you can take advantage of features of XPath, namely count(), substring-before(), substring-after() and all other available XPath functions. Example:

MessageCount=(System.Int32)xpath(CompleteMsg,”count(/*[local-name()=’File’ and namespace-uri()=’http://AppendExample.File’]/*[local-name()=’Record’ and namespace-uri()=’http://AppendExample.Record’])”);

And

FirstName=xpath(IndividualMsg.BodyPart,”substring-before(/*[local-name()=’Record’ and namespace-uri()=’http://AppendExample.Record’]/*[local-name()=’Information’ and namespace-uri()=’’]/@*[local-name()=’Name’ and namespace-uri()=’’],’ ’)”);

LastName=xpath(IndividualMsg.BodyPart,”substring-after(/*[local-name()=’Record’ and namespace-uri()=’http://AppendExample.Record’]/*[local-name()=’Information’ and namespace-uri()=’’]/@*[local-name()=’Name’ and namespace-uri()=’’],’ ’)”);

3.Since there is no such thing as an Envelope Schema that can be used in a Flat File scenario, you can use XPath to extract internally the separate documents and manipulate the data as needed. Example:

xpathString=System.String.Format(”/*[local-name()=’File’ and namespace-uri()=’http://AppendExample.File’]/*[local-name()=’Record’ and namespace-uri()=’http://AppendExample.Record’][{0}]”,CurrentRecord);

tempXML=new System.Xml.XmlDocument();

tempXML=xpath(CompleteMsg,xpathString);

4.You don’t need to have a Construct Shape with an internal Message Assignment Shape, you can simply use the Construct Keyword; you can construct multiple messages at the same time. Example:

construct AfterMsg,BeforeMsg
{
AfterMsg=tempAfterXML;
BeforeMsg=tempBeforeXML;
}

One word of caution though, if you are using the Tracking Profile Editor to capture data, it will not show as an option, so use this only if you are not going to use the TPE.

5.You can take a message and convert it into text for insertion into database columns:

tempXMLData=IndividualMsg.BodyPart;
StringWriter=new System.IO.StringWriter();
XmlTextWriter=new System.Xml.XmlTextWriter(StringWriter);
tempXMLData.WriteTo(XmlTextWriter);
rawXmlData=StringWriter.ToString();

6.Using XPath, you can update attributes/elements. Example:

xpathString=System.String.Format(”/*[local-name()=’Request’ and namespace-uri()=’http://exampleDataStore’]/*[local-name()=’sync’ and namespace-uri()=’http://exampleDataStore’]/*[local-name()=’after’ and namespace-uri()=’http://exampleDataStore’]/*[local-name()=’DataStore’ and namespace-uri()=’http://exampleDataStore’][{0}]/@*[local-name()=’FirstName’ and namespace-uri()=’’]”,CurrentRecord);

xpath(tempDBInsertMsg,xpathString)=FirstName;

Word of caution though: you need to make sure that the attribute/element exists, xpath will not create it, just populate it.

7.Using Microsoft’s built in System.XML functionality; you can append data inside of an already existing xml structure. Example:

internalBeforeXmlDocument=IndividualMsg.BodyPart;
tempBeforeXML=(System.Xml.XmlDocument) tempBeforeXML.CloneNode(true);
XmlNode = tempBeforeXML.CreateNode (System.Xml.XmlNodeType.Element,”ns1″,”Record”,”http://appendexample.record/“);
XmlNode.InnerXml=internalBeforeXmlDocument.FirstChild.InnerXml;
tempBeforeXML.FirstChild.AppendChild(XmlNode);
BeforeMsg=tempBeforeXML;

8.Using Multi-part messages, you can call Business Rules to update ’Context’ data, since the BRC cannot access the Context data, accessing the Context Part of the data serves the same purpose.

HL Numbering

HL Numbering

Back in the day (BizTalk 2002) the incremental numbers in the HL Segments were something that an average developer did not need to worry about.

However, with newer versions comes enhancements, built in tools that make it easier and quicker to develop the same solution. HL Numbering however is actually harder. Microsoft has decided that it is not going to modify any of the data inside of the Transaction using the HIPAA adapter. I had to create a process that would do the numbering.

The trick was to get the numbers to increment regardless of where in the transaction the work was being done. I used some simple C# code to create a constant variable and increment the number.
incrementer.JPG

I created a self contained orchestration that would accept a Claim (institutional or professional) re number it and then send it back. I did this so I can do my mapping logic and then finally plug this in.

Overview of SFTP, FTPS, SCP and FTP

Overview of SFTP, FTPS, SCP and FTP:

I am not going go in detail about sFTP
or FTPS here, since my main motive is to integrate sFTP with BizTalk, hence if
you need more information need to research by yourself


SFTP:

sftp is an interactive file transfer program, similar to
ftp, which performs all operations over an encrypted ssh transport”.

SFTP is part of the OpenSSH standard. Along with SSH you have
capabilities to use commands such as SCP, SFTP, etc. So….SFTP is very secure
because each packet is encrypted and compressed while sending is taking place.
It utilizes public key authentication and compression.
The Abbreviation SFTP nowadays invariably means Secure File Transfer Protocol,
however there is another protocol with the same abbreviation (Simple File
Transfer Protocol). So lets not get confused.

Differences
between FTP, SFTP
and FTPS:

Why not just use FTP? Why should I use Secure FTP instead of
FTP? Why is SFTP better than FTP? <!–[if !supportLineBreakNewLine]–>

FTP Vs SFTP
Vs FTPS Vs SCP:

FTP (File
Transfer Protocol)
does
not take any precautions to protect information transmitted during a session.
This includes your username, password, and any files transmitted.

SFTP
(Secure File Transfer Protocol)
is a secure replacement
for FTP (File Transfer Protocol) based on the Secure Shell protocol. Unlike
FTP, SFTP encrypts both commands and data providing effective protection
against common network security risks. SSH Client and Server provide both
command-line SFTP tools and a graphical user interface for Windows users.

SFTP encrypts the session, preventing the casual detection of your
username, password or anything you’ve transmitted.

The
major reason for implementing SFTP versus FTP is security. FTP is not even
remotely secure. During an FTP session, your username and password are
transmitted in clear text. If someone is eavesdropping, it is not difficult for
them to log your FTP username and password.

In
FTP all data is passed back and forth between the client and server without the
use of encryption. This makes it possible for an eavesdropper to listen in and
retrieve your confidential information including login details. With SFTP all
the data is encrypted before it is sent across the network.

SFTP
is not the same as FTP/S; the latter implements the original FTP protocol
through a separately-created secure tunnel.

FTPS
(FTP-SSL)
is a real ftp that uses
TSL/SSL to encrypt the control session and if required the data session. With
FTPS the control session is always encrypted, but the data session might not
be. Why is this? Because with the control session encrypted the authentication
is protected and you always want this (normal ftp uses clear text). If you are
NOT pre-encrypting the file, you want the data session encrypted so that the
file is encrypted while the data is in flight. However, if you are
pre-encrypting the file then you do not need to have the data connection
encrypted as you do not need to add the overhead of encrypting the data
connection, since the file is already encrypted.
Understand that
SFTP is SSH file transfer and FTPS is FTP with SSL, FTPS
is a file transport layer on top of SSL or TLS.
The FTPS adds
SSL-enabled FTP send and receive capabilities, uses the FTP protocol to
transfer files to and from SSL-enabled FTP servers.



SCP (Secure Copy) is a
non-interactive command-line tool for securely transmitting files from a
machine to another. It is a secure replacement for RCP and provides a similar
command-line syntax. SCP is strongly based on SFTP but is often a more suitable
choice when setting up unattended file transfers using scripts.


 

SSH Servers

OpenSSH (http://www.openssh.com/)

  1. OpenSSH
    is a free version of the SSH protocol suite. OpenSSH servers and clients are
    available for both Unix and Windows.

  2. OpenSSH
    has been ported to Windows with the popular CYGWIN utility. The project was
    created by Network Simplicity (
    http://w