BizTalk Server 2013 R2 Integration with Cloud API Last.fm

BizTalk Server 2013 R2 Integration with Cloud API Last.fm

In previous post I described a way to consume a public Rest API using the BizTalk WCF-WebHttp adapter in combination with JSON-decoder, which is a new component with the BizTalk Server 2013 R2 edition. Now I like to mix things up a bit and consume a different API that is public. That is you can use this API from Last.fm. This is an online music discovery service that gives you personalised recommendations based on the music you listen to. To use the API of this service you need to registering yourself first. Because when you call of one of the methods of the API you need to stick in an api_key as one of the parameters of your request. This is not uncommon as various cloud API’s have this kind of mechanism.

Scenario

I have the following scenario, where I have built a client application (still one of those old fashioned guys that use window forms). The client application have the following functionality:

  • Get information of an artist.
  • Get the top albums from the artist.

Information and top albums can be obtained through calling the Last.fm API artist methods. The client will via BizTalk call these API methods. Similar as in my previous post calling the Airport Service to retrieve its status. Below you find an overview of the scenario.

image

The communication with the internal endpoint in this scenario will be SOAP/XML. The endpoint is hosted in a two way receive port (request/response). It exposes schemas for two operations: GetArtistInfo and GetTopAlbums. The request message will subsequently be mapped to a REST call to the service indicating that the expected response format is Json or default xml. BizTalk will decode the response into XML, so that it is published as an XML message in the MessageBox in case the response message is Json (GetArtistInfo) otherwise it will be just received by the adapter (GetTopAlbums). The receive port will subscribe on that message so it will be routed back as response to the client that on his turn renders it in the UI (Form). This scenario shows that BizTalk acts as a broker and protocol mediator (SOAP/XML –> REST/JSON –> SOAP/XML or SOAP/XML –> REST/XML –> SOAP/XML) between internal client and the external Last.fm API.

The solution of the described scenario consists of the following parts that will be discussed in the subsequent paragraphs:

  • Exposing schema’s for internal service exposing an operation to client application that will consume the service.
  • Creating a custom receive pipeline to enable decoding of Json message to xml (see previous post).
  • Configuration of a Send Port with the Web-Http adapter (or binding if you like), send and receive.
  • Demonstrating the solution.

Exposing schema’s as service

To support both calls from the client to the Last.fm API the request schema’s are as follows:

clip_image003
Both request schemas look the same expect for the root name. These could be consolidated to one schema, nevertheless I choose to keep each method call isolated. Both schema contain promoted properties. The elements need to be promoted to variable mapping later when configuring the send port with WCF-WebHttp adapter to support dynamic URL mapping.

The response for the GetArtistInfo will be Json and therefore I will use the postman application in Google Chrome:

clip_image005

Here you can see that for calling the API you need a method parameter, artist name, api_key and format. However, the format is optional. By default XML will be returned when no format has been specified. The Json response can be used as instance for creating an XSD using the JSON Schema Wizard in Visual Studio BizTalk templates. The schema looks like:

lastfm response schema 1

Similar approach will be used to get an instance of the response to the GetTopAlbums call. This schema will be based on XML. Having the schemas enabled me to create an internal service that exposes two methods.

Once I have the internal service up and running the next part is to create a custom pipeline for receiving the Json response from the GetArtistInfo API method call. The Json decoder will be specified to serialize that response into XML. For the GetTopAlbums no specific custom pipeline is necessary. The schemas and custom pipeline will be deployed to BizTalk runtime.

Creating and configuring the Send Port with the Web-Http adapter

To be able to communicate with the Last.fm API and call both methods I will need to have two send ports configured with the WCF-WebHttp adapter. The Last.fm API doesn’t require authentication other than supplying the api_key as a parameter in call tp any of the API methods. In the general tab of the WCF-WebHttp Transport properties the address of the service can be specified (URI). Besides the address I need to specify here the HTTP Method (GET) and perform a URL mapping.

clip_image008

The URL mapping will be interesting as I need to add a few parameters to my REST call.

http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=Metallica&api_key=<your last fm api_key>&format=json
My HTTP Method and URL Mapping will look like:
<BtsHttpUrlMapping><Operation Method=”GET” Url=”/?method={method}&amp;artist={artist}&amp;api_key={api_key}&amp;format=json”/></BtsHttpUrlMapping>

Interesting thing in this URL mapping is that & is and &amp;. If you try to just use the & you will run into an error like depicted below:

clip_image010

Next I click Edit… to do the variable mapping i.e. map the parameters to promoted properties of my incoming request message.

clip_image011

Variable is mapped to the property namespace that defines the API_KEY, ARTIST and METHOD.
The general tab is important for specifying the address, method and URL mapping. The other tabs are:

  • The Binding tab provides you the ability to configure the time-out and encoding-related properties.
  • The Security tab provide you the ability to define the security capabilities of the WCF-WebHttp send port.
  • The Behaviour tab provides you the ability to configure the endpoint behavior for the send port.
  • The Proxy tab provides you the ability to configure the proxy setting for the WCF-WebHttp send port.
  • The Messages tab provides you the ability to specify how the message is sent to the REST interface.

Note: In this scenario we only use GET operation of the Last.fm API service. Based on the verb you have to specify in the Suppress Body for Verbs the GET, because a payload is not required for this operation. Since BizTalk sends out messages with a payload this needs to suppress!
For further details on configuration settings for these tabs see MSDN article How to Configure a WCF-WebHttp Send Port.

Test the solution

Now building the client to call the Last.fm API methods indirectly via BizTalk was quite some work. I wanted to render the information nicely in a Windows Form. When I enter an artist name and click GetInfo then a request will be send to BizTalk routed to the send port that communicates with Lastfm API and request info of the band Metallica.

clip_image013

The response of the message is nicely rendered in the above form. When I click TopAlbums another request is sent to a different send port that send to a different Last.fm API method.

clip_image015

If we look at the traffic between BizTalk Server and Last.fm using Netmon I can examine what goes over the wire.

clip_image017

This blog has demonstrate how fairly easy it is to consume a Json message with BizTalk Server 2013 R2 after invoking a Rest API. And how I was able to leverage an API from Last.fm. The cool thing is that BizTalk Server 2013 R2 is capable to communicate with tons of REST API’s out there in the cloud with the WCF-WebHttp adapter. And with JSON support things get less complex. I haven’t tried communicating with an API that requires Basic- or OAuth authentication. I probably will have to do some custom coding using behavious like explained in the blog post from Quicklearn.

Cheers,

Steef-Jan

BizTalk Server 2013 R2 Consuming JSON Messages

BizTalk Server 2013 R2 Consuming JSON Messages

BizTalk Server 2013 introduced a couple of new adapters. One of them was the WCF-WebHttp adapter that offers REST Support. The WCF-WebHttp adapter gives you the ability to send messages to Restful services and receive messages through an exposed endpoint. One of the limitations with the adapter (binding) was the lack of Json support. You had to write your own custom pipeline components to serialize the Json format to XML (you can read about it in this blogpost: BizTalk Server support for restful services). In the new BizTalk Server 2013 R2 there is out-of the box support for sending and receiving JSON messages with the following features:
  • a wizard to generate XSD schema from a JSON instance,
  • and an Encoder and Decoder component to use with custom pipelines.

You do not have to write your own custom components anymore. By creating a custom pipeline and dragging either a JsonEncoder or JsonDecoder you can serialize Json into xml or vice versa. With an instance of a Json message you can use the wizard to create a Json XSD schema.

Scenario

There are many web services present that have a REST interface and talk JSON over the wire. The WCF-WebHttp adapter in BizTalk Server 2013 R2 provides means to communicate with these services. There are various scenario’s you can think of to how to demonstrate the functionality of the WCF- WebHttp. In this blog I will demonstrate how to consume a relatively simple Restful Endpoint that you can choose from the Restful Service endpoints of the US Federal Aviation Administration. In this case I will use the Airport Service as an example. The Airport service provides the airport status and delay information from the Air Traffic Control System Command Center (ATCSCC) for every US Airport. It has one endpoint that only supports the GET operation.
The GET request is the fundamental, widely used operation in the REST world. You can simply visit a URL in a browser (or programmatically) and for instance in the case of the Airport Service type the following URL:

http://services.faa.gov/airport/status/SEA?format=xml

The browser will return a machine understandable structured data like below:

clip_image002

In this blog we will not specify the xml format, but the Json format i.e. format=json. There will be scenarios in the real world that services do not support xml format and only communicate through the REST protocol and json format. Let’s assume the airport service only supports json. The URL would look like:

http://services.faa.gov/airport/status/SEA?format=json

The following more advanced scenario describes how the airport service is consumed through BizTalk Server 2013 R2 that receives a request from a client application:

From a client application a request for the status of an airport will be send as a soap/xml message. BizTalk will map this request to a GET operation to the Restful service endpoint. That’s is the incoming message contains the airport code that is marked as property for its schema (xsd). That property will mapped to outgoing request URL to call the Restful endpoint. The endpoint on its turn will process the request and hopefully will provide the status of a given airport based upon the airport code provided within the request URL. The result will be mapped to response message that will be routed back to the client application, where it will be rendered in a Windows Form.

image

The communication with the internal endpoint in this scenario will be SOAP/XML as explained. The endpoint is hosted in a two way receive port (request/response). The message will subsequently be mapped to a REST call to the service indicating that the expected response format is json. BizTalk will decode the response into XML, so that it is published as an XML message in the MessageBox. The receive port will subscribe on that message so it will be routed back as response to the client that on his turn renders it in the UI (Form). This scenario shows that BizTalk acts as a broker and protocol mediator (SOAP/XML à REST/JSON à SOAP/XML) between internal client and the external airport service.

Building the solution

The solution of the described scenario consists of the following parts that will be discussed in the subsequent paragraphs:

  • Exposing schema’s for internal service exposing an operation to client application that will consume the service.
  • Creating a custom receive pipeline to enable decoding of json message to xml.
  • Configuration of a Send Port with the Web-Http adapter (or binding if you like), send and receive.
  • Testing the solution.

Exposing schema’s as service

The first step in this scenario is creating the internal service endpoint based on the following schemas
 
clip_image006

Request schema containing one element, AirportCode, which is promoted as property. Later I will explain, the reason why the AirportCode is promoted. The other schema is based on the Json response of the Airport Service. By calling the service in the browser (Chrome with Postman application) using the following http://services.faa.gov/airport/status/SEA?format=json you can obtain the Json.

clip_image008

Save this file as json. Subsequently you follow the following steps:

  • In the Solution Explorer, right-click the project name > Add > New Item > JSON Schema Wizard. Provide a name for the schema (JSONSchemaAirportStatus.xsd), and then click Add.

clip_image010

  • In the JSON Schema Wizard, on the welcome page, click Next.

clip_image011

  • In the JSON Schema Information page, provide the location of the JSON purchase order file that is sent to the BizTalk Server application. Provide a root node name, a target namespace, and then click Finish.

clip_image012

  • Now you will have a schema like below:

clip_image014

First schema will be request of the service endpoint and the second the response the generated schema based on json. Both schema are with the same BizTalk project. You must compile this BizTalk project as you will need the assembly later.

The following steps will lead to creation of a WCF Service based on the early two created schemas:
Launch the BizTalk Web Services Publishing Wizard and follow the steps described in MSDN page How to Use the BizTalk Web Services Publishing Wizard to Publish Schemas as a Web Service. Basically you launch the wizard, pass the welcome screen. Specify adapter (WCF binding) to communicate with the service, whether or not you want the service expose Meta data, in which application (receive location) you like tie the service to.

clip_image016

  • Click Next and select Publish schemas as WCF Service.

clip_image018

  • Again click Next and start specifying the service operations, assign schema’s you created earlier and that are within the BizTalk assembly to request and response of the Service Method.

clip_image020

  • Again click Next and to specify the location of the service in IIS. BizTalk delegates the hosting of service to IIS, yet to start the service it relies on the receive location to be enabled.

clip_image022

  • Last time to click Next to see the summary of what you have specified for the service.

clip_image024

  • Click Create to provision the WCF-service.

clip_image026

  • Click Finish to end the Wizard. The provisioned service will appear in IIS.

clip_image028

  • When you enable the receive location in BizTalk that has the Uri of the service you can browse to it.

clip_image030

  • The service is up and running the WSDL can be imported as service reference in the client.

clip_image032

Create a custom receive pipeline

BizTalk Server 2013 R2 provides pipeline components that can be used to process JSON messages within a BizTalk Server application, i.e. JSON decoder and JSON encoder. For the custom receive pipeline we will use the JSON decoder pipeline component. To create the custom receive pipeline you can follow the steps below:

In Visual Studio within your Solution Explorer of your project, you right-click and point to Add > New Item > Receive Pipeline. Specify a name for the pipeline name like JsonReceive.btp, and then click Add. Within the Decode stage you can add the new JSON decoder. In the other stages and other pipeline components as shown in the screenshot, and save changes.

clip_image034

In the properties of the JSON decoder you specify the Root Node and Root Node Namespace.

clip_image036

You can do this at design time like above or in run-time.

clip_image037

Next you add the XML Disassembler pipeline component in the Dissemble stage. Save and your custom receive pipeline for serializing JSON to XML is ready.

Creating and configuring the Send Port with the Web-Http adapter

To be able to consume the Airport Service with BizTalk you will need to have a send port configured with the WCF-WebHttp adapter. This can be done by configuring the WCF-WebHttp adapter or binding if you like in case you choose WCF-Custom. The configuration is pretty straight forward. The Airport Service is a public service that requires no authentication for its usage. In the general tab of the WCF-WebHttp Transport properties the address of the service can be specified (URI). Besides the address you specify here the HTTP Method (GET) and perform a URL mapping.

clip_image038

In HTTP Method and URL Mapping section you specify the method (operations) you are going to perform. In the case of the Airport Service this is going to be only the GET. In case you use an orchestration that the Name has to be specified, which is the name of the operation of the logical port. The URL mapping you define what is going to added after the specified URI. To make it more dynamic instead of hard-coding in general or like in this scenario the airport code you make use of variable mapping configuration feature. So what’s between the brackets is a variable that can be mapped to promoted property. The HTTP Method and URL Mapping looks in this case like:

<BtsHttpUrlMapping>
<Operation Method=”GET” Url=”status/{airportcode}?”/format=json”>
</BtsHttpUrlMapping>

Variable mapping is powerful technique to define any custom variable (or place holder) in your URL, in this scenario case {airportcode} and map that variable to any context property with property name and namespace. The Variable mapping is specified by click the Edit… button.

clip_image040

Variable is mapped to the property namespace that defines the AirportCode.
The general tab is important for specifying the address, method and URL mapping. The other tabs are:

  • The Binding tab provides you the ability to configure the time-out and encoding-related properties.
  • The Security tab provide you the ability to define the security capabilities of the WCF-WebHttp send port.
  • The Behaviour tab provides you the ability to configure the endpoint behavior for the send port.
  • The Proxy tab provides you the ability to configure the proxy setting for the WCF-WebHttp send port.
  • The Messages tab provides you the ability to specify how the message is sent to the REST interface.

Note: In this scenario we only use GET operation of the Airport service. Based on the verb you have to specify in the Suppress Body for Verbs the GET, because a payload is not required for this operation. Since BizTalk sends out messages with a payload this needs to suppress!

clip_image042

For further details on configuration settings for these tabs see MSDN article How to Configure a WCF-WebHttp Send Port.

Test the solution.

Last part is the client. The client in this scenario is a Windows Forms application. This application has a simple UI and a reference to the created WCF-Service host in a BizTalk receive location. The client will send the chosen airport code to the service.

clip_image043

The client has a combo box with airport names and corresponding codes of all the US Airports. When you select SEA the code for Seattle/Tacoma Airport then the airport code SEA will be sent as a request message to the WCF-Service. Eventually the response will be rendered in the client.

clip_image044

If you want to monitor the network traffic between BizTalk and the Restful service you could for instance use Netmonitor 3.4.

clip_image046

And examine the out- and inbound traffic from BizTalk to Restful endpoint and back.

clip_image048

In case you enable tracking for the WCF-WebHttp configured send port you can examine the tracked messages in BizTalk.

clip_image050

Above you see the response from the Restful endpoint that arrives at BizTalk.

clip_image052

And how it is send to the client as XML.

This blog has demonstrate how fairly easy it is to consume a json message with BizTalk Server 2013 R2 after invoking a Restful service. Now custom coding is required to serialize a Json format into XML the internal format of the BizTalk messaging engine. Currently REST/Json has taken over the XML/Soap world, at least in the cloud that is. Numerous services available in the cloud support REST let alone only support REST. Therefore, BizTalk Server has adapted to that shift in cloud and supports REST through the WCF-WebHttp adapter and support for JSON.

Cheers,

Steef-Jan

Looking back at 2014

Looking back at 2014

Close to  the end of the year 2014. It has again been quite a year from me, travelling to different places of our world. Some familiar and some new places. 2014 brought me to the following cities, area’s and countries:

  • United-Kingdom London.

London 1

London 2

  • Ireland Dublin.

Ireland 1

  • Italy Tuscany, Emilia Romagna and Lombardia (Pisa, Lucca, Viareggio, Florence, Piacenza, Poggi Bonsi, San Gimignano, Volterra, Sienna, Valconasso di Pontenure, Podenzano, and Val Trebia).

Italy 3

Italy 2

  • United-States Washington State (Seattle, Bellevue, Redmond, Deception Pass, Whidbey Island, Fidalgo Island, Camano Island, and Bainbridge Island).

BizTalk Summit 4

WP_20141206_074

  • Malaysia, Kuala Lumpur.

Kuala 1

  • Australia Victoria, New South Wales and Queensland (Melbourne, Brisbane and Sydney).

Aus 9

Aus 1

  • Spain Basque Country, Spanish Pyrenees (Girona, L’estartit, Vielha).

Spain 1

  • Norway Olso, Bergen.

Norway 2

  • Sweden Stockholm.
  • Belgium, Mechelen.

For one it has been a very busy year like last year 2013. To summarize all my activity bullet wise:

  • I have had multiple speaking engagements in the Netherlands and abroad (UK, Norway, Sweden, Belgium and Australia).

Aus 6

  • Attended the annual MVP Summit (November).

BizTalk Summit 1

BizTalk Summit 2

  • Attended the BizTalk Summit (INTEGRATE2014) in Redmond (December).

INTEGRATE 2014 2

INTEGRATE 2014

  • Technical reviewer for a number of books and white papers.

WP_20141205_065

  • Watched dutch soccer perform well during the WorldCup 2014.

Soccer 1

  • Written multiple TechNet wiki articles on BizTalk and ran two half Marathons, The Hague and Berlin, in March.

Marathon 1

  • Written dozen blog posts on this blog (especially liked the ones on service virtualization I created with the help of Andrew Slikver/Nevatech) and on the TechNet Wiki Blog.

For me personally I am happy to finally been to Australia and had a great time speaking at three cities. I really like to thank Saravana Kumar, Dean Robertson, BizTalk360 and Mexia to make this possible. Both companies made a huge commitment and made a tremendous investment in the community. Besides BizTalk360 and Mexia there are some other companies that have shown a lot of affection to our integration (BizTalk) community like: Codit, Quicklearn and Bouvet.

Aus 8

Aus 5

After the event I had a great time with my friends Mikael Hakansson and Mick Badran. Both showing me the Aussie way of live. Thanks guys!

Aus 3

I am also happy to see that both Richard Seroter and Stephen W. Thomas, two of my other friends, welcomed a healthy, lovely daughter in their family lives: Charlotte and Haydon.

To conclude I have had an amazing year spending working on several integration projects, travelling, seeing the Seahawks become SuperBowl champs, seeing my kids grow up, spending time with the family, friends and of course the BizTalk community. Thanks everyone for reading my blog and I will continue to share my knowledge through speaking and writing in 2015.

WP_20141204_003

Cheers,

Steef-Jan

INTEGRATE 2014 – Journey to next generation of integration

INTEGRATE 2014 – Journey to next generation of integration

The beginning of december I visited the Microsoft Campus (Building 92) for the 3rd global BizTalk Summit, which was branded as Integrate2014. The event was organized this time by BizTalk360  along side with Microsoft itself and core partners like Quicklearn, Nevatech, Codit, and LogicalTech SysTalk. It was a three day event filled with numerous sessions by Microsoft themselves, a few MVP’s and partners.

WP_20141202_16_45_03_Pro__highres

The first important message to me, partners and end-users I believe was that BizTalk Server is going to play an essential role for integration systems and applications on-premise. Besides that its also provide support with connectivity to cloud (Azure with adapters that support SB-connectivity, and relay). That means BizTalk Server can also provide enterprises the ability to build cloud to ground or ground to cloud based solutions (see also Richard Seroter’s course on Pluralsight called Cloud Integration Patterns).  The product group explicitly shared that there will be future platform upgrades (full version, or R2). See also my post on the TechNet Wiki Blog: The Future of Integration.

WP_20141204_013

Second even more important message was the new direction for BizTalk Services. MABS 1.0 will change course to adopt architecture called MicroServices. This means that MABS 2.0 will fundamentally change to support this architecture. What the architecture basically boils down to is that granularity will increase. A BizTalk Service solution can consist of many small services like connectors (called triggers and actions), mediation (that supports VETER pattern we know from MABS 1.0), BPM, Mappings, and Business Rule Engine (BRE). During the event we have seen different sessions on MicroServices from different speakers. All sessions were recorded and will be made available soon.

WP_20141205_009

Finally the number of attendees and partners (sponsors) showed me that ecosystem of Microsoft Integration is very alive. Microsoft will continue to innovate it’s on-premise product BizTalk Server and it’s cloud based offering BizTalk Services, which will be named Azure BizTalk Microservices (MABS 2.0). We stand before the door to the next generation of integration solutions. We slowly we move (migrate) integration solution to Azure Platform and build new solutions to communicate with distributed systems (anywhere, anyplace, anytime) and devices.

WP_20141205_069

Other blog post on INTEGRATE 2014 definitely worth to checkout are:

It has been an amazing event and it was awesome to meet with people like Microsoft Integration MVP’s present, Microsoft BizTalk Product Group, Biztalk360 Team, Quicklearn Team, Andrew Slivker (Nevatech), Jason Sander, Dwaine Gilmour, Mandi Ohlinger, Richard Broida, Tom Canter, Rob Fox, Jean Paul Smit, Martin Peters, Rene Brauwers, JoAnn Een, Michael Shea, Koen van Oost, Eldert Grootenboer, Rex van der Laan, Bart Scheurweghs, Rajesh Kolla, Howard Edidin, and list goes on.

WP_20141204_022

BizTalk360 as event organizer have done an outstanding job taking the BizTalk Summit (Integrate) to the next level!

Cheers,

Steef-Jan

BizTalk360 – An easy way to evaluate

BizTalk360 – An easy way to evaluate

BizTalk360 is a full featured BizTalk Server Monitoring & Operations product. The product has matured from a small web based monitoring tool to a full blown feature rich product for monitoring BizTalk Server. The product includes features for BizTalk Operation support people like the Throttling analyzer, Topology Viewer, and Backup DR Monitor. Recently a new version 7.8 has been released with another set of valuable features:

  • Advanced Process Monitoring and dashboard
  • EDI reporting capabilities in line with BizTalk Admin Console.
  • System Profile: Ability to remove features you don’t want in BizTalk360.
  • User Profile: Flexibility to set custom time zone and date/time format for each user.
  • Removed Orphaned artifacts from Monitoring: Ability to remove old orphaned artifacts like Applications, which no longer exist in the environment.

If you haven’t heard of BizTalk360 than you will like this blog post as it will explain how you can easily evaluate the tool by yourself and witness the great value this product can offer. Now this will be not an extensive post to showcase each feature. I will focus on how you can leverage the features: Graphical Flow (Tracking), Business Activity Monitoring (Portal), and the Tracking Manager.

Easy way to see these features in action is by deploying the BamEndToEnd example you can find in the <Samples Path>BamEndToEndInput folder of the installation of your BizTalk Server. This sample stems from version 2004 and can be used from 2006 up to 2013 R2. The demonstrations in this post is the BamEndToEnd example with BizTalk Server 2013 R2, and the latest BizTalk360 Version 7.8.

After deploying the BamEndToEnd example you will need to create the tracking profile yourself and deploy it. Next you will to add/verify the tracking is enable on the send, receive ports and the orchestrations.

Note that tracking can be very useful for troubleshooting and traceability in general. However in production environments you beware that extensive tracking can influence performance. For best practices for tracking see blog post on BizTalkAdminsBlogging Best Practices for Tracking. With regards to orchestration tracking is that Event Tracking is enabled by default. This can be helpful during development, testing and troubleshooting with the Orchestration Debugger. In Production, these tracking events should be disabled, because these events are written to the TrackingData_x_x tables in the BizTalkMsgBoxDb database and then moved to the dta_DebugTrace table in the BizTalkDTADb database by the Tracking host. If the TrackingData_x_x tables (BizTalkMsgBoxDb database) and/or the dta_DebugTrace table (BizTalkDTADb database) get too large*, performance will be impacted, see How to maintain and troubleshoot BizTalk Server databases.

clip_image002

After deployment and configuration is done you can drop the test messages in the folder belonging to the receive location. In SQL Studio Manager you can execute the query below to examine if data is present in the BAMPrimairyImport database tables belonging to the BamEndToEndActivity.

clip_image004

When that is in order you can switch to BizTalk360 product. The product is available for free trail. The installation is straightforward described on the site on this page. Once you have installed and configured BizTalk360 you can start looking at some of the features. One is the Graphical Flow (Viewer). You can select this viewer and leave everything default and click Execute Query.

clip_image006

Next you can select one of the tracked service instances.

clip_image008

On right hand side you can see the properties of the third orchestration of the BAMEndToEnd example. You can click the blue dot (where the message is received) and see a line going from the green dot (message being send from a location) of the second orchestration. Now you can click in the right hand pane the context and/or content tab. You will see the context of the message going from Orchestration 2 to 3 and the content.

clip_image010

You can click the blue dot again and reach orchestration 1 the first orchestration message are flowing through. This feature is a very valuable feature in case you need to troubleshoot a process that for instance consists of multiple orchestrations. From a messaging perspective you can look at messages coming in and out of BizTalk. In the dashboard you can select Tracked Message Events instead of Tracked Service Instances. Select one of the Send Type the eye icon.

clip_image012

A dialog will appear for the message you selected. You can see the general properties, context properties and the content. Note this depends on how tracking is configured. For both Tracked Message Events and Tracked Service Instances you add filters that can enhance search ability for a certain message(s) and instance(s).

Next we will look at the Business Activity Monitoring. BizTalk360 offers a view/dashboard for BAM that is similar to the BAM Portal. One of the concepts of BizTalk360 is offering a single operation and monitoring experience. This means one product for accessing data and user experience (UI) for BizTalk, BAM, BRE, EventViewer, BizTalk databases and ESB Toolkit.

clip_image014

If you go to Business Activity Monitoring Dashboard in BizTalk360 you can do similar things one can do in the BAM Portal. In The BAM Dashboard you can select a BAM View. In this case the BAMEndToEnd View. Select columns and execute a query.

clip_image016

As you can see the data can be viewed from the dashboard. There is no need to go to a separate Portal in case your BizTalk solutions include BAM. The BAM Dashboard in BizTalk360 will also give you a view of the sizes of the BizTalk BAM Databases.

clip_image018

Sizes of your BAM databases matter. In case BizTalk system is under a huge load a significant amount will be persisted to the BAM database. Therefor information on size and growth are valuable. You can monitor this by setting alarms in BizTalk360.

The final feature I like to discuss in this context is the Tracking manager. For the BAMEndToEnd example you can enable tracking on send/receive ports and the orchestrations. By selecting the Tracking Manager in BizTalk360 you can examine how tracking is enable on the receive ports, send ports and orchestrations.

clip_image020

You can select different applications in this dashboard, examine if global tracking is enabled and how per application the tracking is set on receive/send ports and orchestrations. This manager or dashboard if you like gives you a quick view on how tracking is set up in your BizTalk environment. This can be useful in cases you have not direct insight of the bindings of the applications that are being deployed. The bindings can contain tracking settings, see for instance binding collection of a receive port on MSDN.

In this post I quickly showed three powerful features in BizTalk360. However, there are many more which you can examine by yourself or by reviewing the feature tour on the BizTalk360 site. In case you like to further explore the features BizTalk360 offers you can do so by using the BizTalk out-of-the box examples on a VM and the evaluation edition of the product.

Cheers,

Steef-Jan

Going down under–BizTalk Summit 2014

Going down under–BizTalk Summit 2014

There will be another BizTalk Summit event in august in Australia. The early bird tickets are on sale now until the end of this month! The event will take place in three different cities: Melbourne, Sydney and Brisbane.

image

I will be speaking at each of the events together with fellow MVP’s Saravana Kumar, Michael Stephenson, Bill Chesnut, Mikael Hakansson and Mick Badran on various topics around integration with the Microsoft technology stack: Microsoft Azure, BizTalk Service, BizTalk Server 2013 R2, Mobile, Office365, and BizTalk360.

My talk will specifically go into Microsoft Azure BizTalk Service, what it offers today, what scenarios can be supported, development-, deployment-, and operation perspective of this cloud service. Session is called:

Hitchhiker’s guide to integration with Microsoft Azure BizTalk Service

and the actual abstract:

Microsoft Azure BizTalk Service (MABS) is almost available to us for a year. It is a newcomer in the world of integration Platform as a Service (iPaaS) and promising contender for the sweet-spot in the Gartner’s Magic Quadrant. This service in Azure has a great deal to offer and can provide different means of integration in the changing IT world of on premise ERP systems, services, cloud and devices. In this session the audience will learn where the Microsoft Azure BizTalk Service stands in the middle of the iPaaS world, what it has to offer today and what the road-map will look like in the future. During the session there will be various demo’s showcasing the service features from a development, deployment and operations perspective.

You will find the agenda, other speaker bio’s, sessions and abstracts on the site, where you can register.

Looking forward to speak there and meet people in Australia. I have not seen this part of the world yet so it will be quite an adventure. See you there!

Cheers,

Steef-Jan

Microsoft Integration MVP 2014 – 5th Time in a row!

Microsoft Integration MVP 2014 – 5th Time in a row!

Today I have received an e-mail from Microsoft with exciting news that my MVP status has been renewed again!

[MVP_Horizontal_FullColor%255B4%255D.png]

For me this is the fifth time to receive this award. The fourth year in the program has been an awesome experience, which gave me the opportunity to do great things and meet inspiring, very skilled people. I have had some interesting speaking engagement, which were fun to do and were very fulfilling. I learned a lot through speaking thanks to the community and fellow MVP’s. I was able to share my experiences through these speaking gigs and other channels like this blog, MSDN Gallery, and above all the TechNet Wiki.

I would like to thank:

  • My MVP leads William Jansen.
  • The BizTalk Product Team, Mark Mortimore, Guru Venkataraman, Karthik Bharathy, Ed Price, Nitin Mehrota, Mandi Ohlinger, Allesandro Teglia, Paolo Salvatori, Clemens Vasters and all other Microsoft employees involved.
  • People at my former employer motion10: Rene Brauwers, Eldert Grootenboer and many others. 
  • At my current company Inter Access: Fellow MVP Edward Bakker.
  • The BizTalk Crew: Saravana Kumar (BizTalk360), Nino Crudele, Sandro Pereira, and Tord G. Nordahl
  • Fellow Microsoft Integration MVP’s: Richard Seroter, Kent Weare, Mikael Håkansson, Johan Hedberg, Stephen W. Thomas, Mick Badran, Micheal Stephenson, Jon Fancey, Tomasso Groenendijk, Dan Rosanova, Ben Cline, Sam VanHoutte, Bill Chesnut, Leonid Ganeline, Ashwin Prabhu and Maheshkumar S Tiwari, who I got learn even better and supported me in this program.
  • The BizTalk community: Mikael Sand, Lex Hegt, Colin Meade, Naushad Alam, Howard S. Edidin, Johann Cooper, Mitch VanHelden, Jérémy Ronk,  Maxime Labelle, Jean-Paul Smit, Dean Robertson, João Pedro Martines, Martin Abbott, and many others that make the BizTalk community strong! 
  • Andrew Slivker from Sentinet.
  • Finally my wife Lian and children Stan, Ellis and Cato for their support.

I’m looking forward to another great year in the program.

P.S: Four years ago when I first received this award the Netherlands did not win the World Cup final. Hopefully this time in 2014 they will reach the final and win.

Cheers,

Steef-Jan

Microsoft Integration MVP 2014 – 5th Time in a row!

Microsoft Integration MVP 2014 – 5th Time in a row!

Today I have received an e-mail from Microsoft with exciting news that my MVP status has been renewed again!

[MVP_Horizontal_FullColor%255B4%255D.png]

For me this is the fifth time to receive this award. The fourth year in the program has been an awesome experience, which gave me the opportunity to do great things and meet inspiring, very skilled people. I have had some interesting speaking engagement, which were fun to do and were very fulfilling. I learned a lot through speaking thanks to the community and fellow MVP’s. I was able to share my experiences through these speaking gigs and other channels like this blog, MSDN Gallery, and above all the TechNet Wiki.

I would like to thank:

  • My MVP leads William Jansen.
  • The BizTalk Product Team, Mark Mortimore, Guru Venkataraman, Karthik Bharathy, Ed Price, Nitin Mehrota, Mandi Ohlinger, Allesandro Teglia, Paolo Salvatori, Clemens Vasters and all other Microsoft employees involved.
  • People at my former employer motion10: Rene Brauwers, Eldert Grootenboer and many others. 
  • At my current company Inter Access: Fellow MVP Edward Bakker.
  • The BizTalk Crew: Saravana Kumar (BizTalk360), Nino Crudele, Sandro Pereira, and Tord G. Nordahl
  • Fellow Microsoft Integration MVP’s: Richard Seroter, Kent Weare, Mikael Håkansson, Johan Hedberg, Stephen W. Thomas, Mick Badran, Micheal Stephenson, Jon Fancey, Tomasso Groenendijk, Dan Rosanova, Ben Cline, Sam VanHoutte, Bill Chesnut, Leonid Ganeline, Ashwin Prabhu and Maheshkumar S Tiwari, who I got learn even better and supported me in this program.
  • The BizTalk community: Mikael Sand, Lex Hegt, Colin Meade, Naushad Alam, Howard S. Edidin, Johann Cooper, Mitch VanHelden, Jérémy Ronk,  Maxime Labelle, Jean-Paul Smit, Dean Robertson, João Pedro Martines, Martin Abbott, and many others that make the BizTalk community strong! 
  • Andrew Slivker from Sentinet.
  • Finally my wife Lian and children Stan, Ellis and Cato for their support.

I’m looking forward to another great year in the program.

P.S: Four years ago when I first received this award the Netherlands did not win the World Cup final. Hopefully this time in 2014 they will reach the final and win.

Cheers,

Steef-Jan