BizTalk Sftp Adapter – Troubleshooting

by | Mar 5, 2008 | BizTalk Community Blogs via Syndication

We’ve used and tested this adapter extensively for my former customer, and when correctly configured, we haven’t had any major issues. However, you might find it challenging to use due to other factors such as generating identity key files, opening of firewalls, installing Sftp servers etc.

The exchange of keys is a particular annoying topic. If you are already familiar to the Sftp protocol, you might not see this as an obstacle, but generally when using BizTalk you’d be given any necessary authentication information, and from there you’d go on and configure your port.
In the scenario of using a Sftp adapter, BizTalk would be the client, and would therefore have the private key (in contrast to ssl, where the host has the private key). The owner of the host system will require a public key from you, to create a user account. When given that account, you’re now able to configure and test your adapter.

This might not seem as a problem, but in my experience it is. The fact that the host system ower (if he or she exists), is often not the person who would do the actual work, which means you’d have yet another (Unix) person to talk to. This will increase the possibility of anything going wrong.

Of cource you may use user name/password as authentication, in case you can disregard all I've said… 

Rather than using BizTalk, you may find it easier to use a console application to make sure the connection is working. You could also use other applications such as Tunnelier, for the same reason, but it might work differently. Below you find a simple sample of what the code could look like. Make sure to reference the Blogical.Shared.Adapters.Sftp assembly:

class Program 
    { 
        static void Main(string[] args) 
        { 
            string host = System.Configuration.ConfigurationSettings.AppSettings["host"]; 
            string user = System.Configuration.ConfigurationSettings.AppSettings["user"]; 
            string pass = System.Configuration.ConfigurationSettings.AppSettings["pass"]; 
            string keyfile = System.Configuration.ConfigurationSettings.AppSettings["keyfile"]; 

            using (Sftp sftp = new Sftp(host, user, pass, keyfile)) 
            { 
                try 
                { 
                    sftp.Connect(); 
                    Console.WriteLine("Connected"); 
                    sftp.Disconnect(); 
                } 
                catch (Exception ex) 
                { 
                    Console.WriteLine("-=<>=- EXCEPTION"); 
                    Console.WriteLine(ex.Message); 
                    Console.WriteLine(ex.StackTrace); 

                    int i = 1; 
                    while (ex.InnerException != null) 
                    { 
                        ex = ex.InnerException; 
                        Console.WriteLine("-=<>=- INNER EXCEPTION " + i++); 
                        Console.WriteLine(ex.Message); 
                        Console.WriteLine(ex.StackTrace); 
                    } 
                } 
            } 

            Console.WriteLine("Test completed. Press Enter to Exit application."); 
            Console.ReadLine(); 
            Console.WriteLine("Terminating..."); 
        } 
    }

Another thingthe UseLoadBalancing property on the receive endpoint is by default set to true. This is going to cause you problems if you haven’t created the database table. Make sure to set it off, if you are not going to use it.

BTW, Thank you Johan for providing the code sample above…

BizTalk Database Architecture Poster Now Available

by | Mar 4, 2008 | BizTalk Community Blogs via Syndication

The BizTalk product team keeps pumping out posters, so I guess that I’ll keep highlighting them.  This particular one focuses on the core BizTalk databases, their functions, and how they relate to each other.  Specifically, you’ll find each BizTalk database alongside it’s role, functional events, data movement events and more.  Given that SQL Server databases […]

Rest In Peace Gary Gygax

by | Mar 4, 2008 | BizTalk Community Blogs via Syndication

I

am deeply saddened to learn that today we

have lost Gary Gygax (via

EnWorld), creator of Dungeon’s

& Dragons, from our company in this world. I did not know him personally,

but I can say without a doubt that this man has touched countless people through his

games, and his ambitions.

I starting playing role-playing games when I was 13, having been introduced by a family

friend to what is now affectionately referred to as “Red Box” D&D. In the 21 years

since that time I’ve played many games, I have lived through the turmoil of the “D&D

is satanic” times, but I still to this day play role-playing games on a weekly basis

and I recognize that Gary Gygax’s games helped me define my personality, my sense

of humor, and my style of storytelling. I have in fact just returned from D&D

Experience, one of many large gatherings of those who play D&D still and who participate

in events run by the RPGA and

was learning about the upcoming Fourth Edition of the game.

Gary has not been involved with D&D directly in many years, and the game has moved

from being owned by TSR, to Wizards

of the Coast, and now corporately Hasbro though still through the Wizards of the

Coast brand. I would invite everyone reading to stop and think about the impact

this man has had on our world. While not alone, he is one of the pillars who

brought about Role Playing Games (RPGs) in the United States. From those games

come so much we take for granted, it is from those games we have countless novels,

movies, and computer games as direct products, but also indirectly games such as World

of Warcraft and Lord

of the Rings Online cannot help but acknowledge that the systems used to model

those virtual worlds stand on the shoulders of D&D, and as such this man.

While I did not know him personally, I believe it is safe to say that Gary Gygax remained

a fan of RPGs until his last breath on this planet, and as such I’d like to take a

moment and encourage everyone here to block out some time on June 7th or June 21st

to try your hand at the game again.

  • June 7th, 2008 is the Worldwide

    Dungeon & Dragon’s Game Day where game stores world wide will be opening their

    doors and running games for any who will be interested for free. This is scheduled

    for the day after the release of Dungeon & Dragon’s Fourth Edition and will be

    a great chance to see how Wizards of the Coast have stewarded the brand that Gary

    started so many years ago.

  • June 21st, 2008 is Free

    RPG Day, and many of those same retailers will again be running tables of games,

    this time from a wide variety of game developers not just Wizards of the Coast.

    There will be free games, and gaming supplies, being given away in most of those stores

    that day.

So I raise a toast Gary today, and send my well wishes to his wife, six children and

extended family. And, perhaps belatedly, I thank him for what he has done for

my life without ever knowing it.

Tim Rayburn is a consultant for Sogeti in the Dallas/Fort

Worth market.

Xref Information further explained

by | Mar 4, 2008 | BizTalk Community Blogs via Syndication

A friend asked a few questions about the Xref functiods that I thought everyone (including myself) could benefit from

  1. Can we have many [sic] application using Cross-referencing functoid and pointing to the same database? Yes, this is the beauty of the Xref functionality, you can create multiple Application Types, and application type would be for inbound purchase orders for example, and you can have another application type for outbound invoices for example. Even to a greater extent, you can have application instances for each application type, so you can have a partner specific application instance of the inbound purchase order, and a partner specific application instance for outbound invoices. The ListOfAppType.xml would look like this: 
    <?xml version="1.0" encoding="utf-8"?>
<listOfAppType>
    <appType>
        <name>Inbound Purchase Order</name>
    </appType>
    <appType>
        <name>Outbound Invoice</name>
    </appType>
</listOfAppType>

    And here is the example of how the ListofAppInstance.xml would look:

    <?xml version="1.0" encoding="utf-8"?>
<listOfAppInstance>
    <appInstance>
        <instance>Partner 1</instance>
        <type>Inbound Purchase Order</type>
    </appInstance>
    <appInstance>
        <instance>Partner 2</instance>
        <type>Inbound Purchase Order</type>
    </appInstance>
    <appInstance>
        <instance>Partner 1</instance>
        <type>Outbound Invoice</type>
    </appInstance>
    <appInstance>
        <instance>Partner 2</instance>
        <type>Outbound Invoice</type>
    </appInstance>
</listOfAppInstance>

    Here is what it looks like within the tool

  2. When is purge data script useful? There is a stored procedure that is called xref_Cleanup, this is a stored procedure that brings back the various tables to factory settings. Only use this when you want to clear out everything. There is currently no stored procedure nor out of the box way to clear out portions of the data. I actually opened a ticket because I could not load data after I truncated all of the data in the various tables because I could no longer import data into the tables anymore. Truncating the tables is not the way to clear out the tables, you need to run this stored procedure. For your information, here is the stored procedure to see what it does:

    CREATE PROC [dbo].[xref_Cleanup]
AS
TRUNCATE TABLE [dbo].[xref_AppInstance]
TRUNCATE TABLE [dbo].[xref_AppType]
TRUNCATE TABLE [dbo].[xref_IDXRef]
INSERT INTO [dbo].[xref_IDXRef] (idXRef)
VALUES (N'')
TRUNCATE TABLE [dbo].[xref_IDXRefData]
TRUNCATE TABLE [dbo].[xref_MessageArgument]
TRUNCATE TABLE [dbo].[xref_MessageDef]
TRUNCATE TABLE [dbo].[xref_MessageText]
TRUNCATE TABLE [dbo].[xref_ValueXRef]
INSERT INTO [xref_ValueXRef] (valueXRefName)
VALUES (N'')
TRUNCATE TABLE [dbo].[xref_ValueXRefData]

BizTalk RFID Solution Days Wrap Up (and a some play time)

BizTalk RFID Solution Days Wrap Up (and a some play time)

by | Mar 4, 2008 | BizTalk Community Blogs via Syndication

I’ve now managed to get a moment to share a fantastic experience with you.

What a trip!!! I was away for 2 weeks in total with the first week spent in San Jose
at the Office Developer Conference (ODC) 2008 which as good to touch base with some
of the Sharepoint folks and see how others solve challenges that drive me up the wall.

Being in a hotel in the middle of ‘Silicon Valley’ Clayton and I were struggling to
find Wireless internet access – the frustrating thing was that everywhere else in
the town there were at least 20 networks on offer….except in our hotel room. We
found a spot on the window ledge.

Then I flew up to Seattle to meet up with my enthusiastic cousin which resulted in
a sneaky trip up to Whistler skiing smile_regular

The snow was absolutely fantastic!!!!>

>

The second week of my trip turned to the Business end where I had to step up, be prepared
and present.
Enter BizTalk RFID Solution Days!!!

The first face I saw when I walked into the Conference was Anush’s down in the foyer
– Well done Anush for such a great conference!

The way the week was to play out:

  • First two days was the conference with sessions and presentations
  • The second two days was the training where I had 102 students and fortunately we
    planned that I was co-presenting with Venkatesh who
    was the BizTalk RFID Product Architect (he’s moved on now and started S3Edge.com).
    He is a wealth of knowledge and a great person to ask all the ‘Why’ questions to.
    (he’s also related to a great friend of mine here in Australia which was a total surprise!!!)

From the conference (2 days prior to training) I caught up with a buddy of mine Scott
Allen (Microsoft App Plat TS – BI, BTS, SQL guru) who has just successfully
completed building a ‘RFID Lab’ within Microsoft NZ. I’ve got to check it out sooner
rather than later – sounded fantastic.

Scotty inspired me with his take on BizTalk RFID – he said “If you tell clients we’ve
got a show on BizTalk RFID, they think they need to have supply chains and infrastructure
– it becomes very specialised they feel. If you pitch BizTalk RFID at
something around How to reduce the carbon footprint of your production line or increasing
production efficiencies then the whole world comes running”

That said – he was getting huge amounts of CEOs from the big end of town to these
sessions with a tour of the BizTalk RFID Lab. Well done Scotty!

So Anush grabbed center stage at the beginning of the conference and there were some
great messages coming out of the speakers. One of the most noticeable messages for
me was “Everyone is waiting for the Wallmart mandates to drive the RFID space,
that may have been the case 2 yrs ago. Now we have reached enough critical mass in
the industry to really see the explosion in RFID that is occurring“

>

Australia has been using RFID for years in cattle, lifestock, shipping and now wheat.
What Microsoft BizTalk RFID has done is to ‘domesticate’ the whole process of getting
connected and responsive. I was talking to a local CEO of a International Meat exporting
business on the weekend – he was very interested in the BizTalk RFID story as his
main concern facing his company is traceability. Everyone involved
in the supply chain wants traceability – by the time the consumer sits down and uses
the product, they want to know everything about it. In this case the meat.

Onward to the Training Arena (held on the MS campus) – training 102
people on Microsoft BizTalk RFID I needed to be prepared the best I could which involved
a couple of trips to the MS Technical Services Group (TSG) to get things ‘just right’
for classroom setup.

Firstly – I want to say “WELL DONE TO ALL THE CREW who sat the course”. In
12+yrs of teaching, I was amazed at your ability and enthusiasm!!!

(We actually created the RFID course from scratch and it’s amazing to see what’s in
your head play out better than you expect)

I made a pledge to the class from the outset – From 102 students, I was going endeavor
to help get 102 students across the line so they all had working BizTalk RFID
Solutions in front of them.

What a great class – I enjoyed it as much as what they did. Out of 102 students I
fully expected some students to come up to me by lunch time on the first day and say
“Mick, you know this course – it’s not what I envisaged etc etc” – 102 completed the
course!!!

Let me share some interesting facts:

  1. 102 students started…. 102 students completed a working BizTalk RFID end-to-end
    solution. smile_teeth
  2. Some students had 1+yrs experience in BizTalk RFID…..others were seeing it for the
    first time.
  3. One lady hadn’t cut a line of code in 15yrs and was very excited.
  4. Many different industries were represented in the room, from medical foundations to
    services to agricultural…..
  5. I had to run around with a USB key only once – fortunately I had help.
  6. One the first day students were still doing labs at 7pm.
  7. Two days is not long enough 🙂
  8. It was absolutely fantastic to have the major players from the BizTalk RFID Product
    group there in the room and on hand. Anush, Sudhir, Rama and of course Venkatesh (who
    has a great needed sense of humour for a trainer) – big thank you guys. Wouldn’t
    have been the same without you    .
  9. Reporting Services reports worked really well…..note – getting the right connection
    string is key.
  10. Mick – please connect power to your laptop when presenting and running VPCs off it…..otherwise
    we all get an early coffee break smile_omg

Fantastic and a big thank you to all that were responsible for much
making it better than I imagined!!!

ESB Guidance 101: File receive, dynamic endpoint resolution via rules engine

by | Mar 3, 2008 | BizTalk Community Blogs via Syndication

There was a message at the Codeplex site from someone having issues doing a simple “receive file from folder, do dynamic endpoint resolution and deliver message”. I threw together a little sample showing how this works, and blogged about it in case others need this too.

The moving parts involved are:

  • receive location using the ItineraryReceivePassthrough pipeline
  • dynamic send port using the ItinerarySendPassthrough pipeline

How it works:

  • the message is received
  • the ESB Dispatcher Disassembler calls the Resolver specified in the Endpoint (in this sample, the business rules engine, see below)
  • the Resolver invokes the rules policy
  • the AdapterProvider framework sets appropriate outbound adapter information
  • the message hits the MessageBox
  • a send port subscribes to all messages from that specific receive location
  • the message is delivered

The receive configuration looks like this:

(full endpoint value is “BRE:\\policy=Microsoft.Practices.ESB.Test;version=;useMsg=;MessageExchangePattern=;”)

Next, using the BizTalk Admin tool, right-click your test application and add a reference to the Microsoft.Practices.ESB BizTalk application. That way you’ll be able to access the resources it contains, including the pipelines. We will need this for the next step.

The send side looks like this (note that there are no values set in the per-instance properties):

Then, of course, we need a filter condition to link them together:

The ultra-complex rules policy 🙂 looks like this:

Technorati Tags: ESB Guidance,BizTalk,ESB