Creating Linked Server to Informix

I was given the task of connecting BizTalk to an Informix database. When the project was first kicked off, the client started exploring their options, one of the companies that they looked at was Attunity.

I told them that the Linked Server option was a much better approach, while offering significant cost savings!

So I started my research on how to get connected, and I came across a few pages of interest:

The first is IBM documentation page about how to set up the actual connection within Enterprise Manager.

When I first started connecting, I was getting this error:

After doing some more research I found additional articles that were of interest:

One that I thought might be a problem was the permissions for the temp folder as documented on KB814398 to deal with Error: 7399. I set the permissions as documented with the same results.

Another IBM’s documentation page about how to set up the linked server and the fact that you need to install the coledbp.sql on the Informix database so that the OLE DB can actually connect.

Here is what I had setup (of course the actual connectivity is different)

I had read an article about mapping between the local instance of SQL Server and the security account on a remote server, but did not try it.

I finally clicked on the security button and filled in the username and password

It worked!

What I don’t quite understand is the relationship between the Provider string and the security context they are the same, but when would they be different?

BizTalk Ranger Blog – Paolo Salvatori

My friend and colleague Paolo Salvatori has just started a blog at http://blogs.msdn.com/paolos/ and began with a great post on Synchronous to Asynchronous Flows Without an Orchestration http://blogs.msdn.com/paolos/archive/2008/07/21/msdn-blogs.aspx.  As you’ll see from the post Paolo loves to delve into technical issues and always provides comprehensive report and great source code samples to back up his findings.


I’d definitely recommend adding this blog to your feed/favourites as I can guarantee from Paolo’s previous work that you will see lot’s of interesting content.

BizTalk 2006 R2 Hyper-V Guide

I’ve just completed the BizTalk 2006 R2 Hyper-V Guide project working with the BizTalk UE team and Clint Huffman from Premier Field Engineering. The purpose of this guide is to provide you with the information, techniques and best practices necessary to evaluate BizTalk Server performance in a Hyper-V environment.


 


Overall the results were very positive.  In most scenarios we were able to process about 75% of the load in a virtualized environment compared to an equivalent physical environment. The Hyper-V team have done a great job at improving this when compared to Hyper-V.  Pass-through storage provides fast storage access and they have done a great job at ensuring that CPU and memory overhead of virtualization is kept to a minimum.  You can even install the Hyper-V role on Server Core which minimizes the overhead further.


 


The guide provides full details of our experiements, results and findings.  I hope that you find this information useful.


 


The guide is available on MSDN (http://msdn.microsoft.com/en-us/library/cc768518.aspxand TechNet (http://technet.microsoft.com/en-us/library/cc768518.aspx )


 


 


Launch Mail 


 


The BizTalk Customer Advisory Team and BizTalk UE team are pleased to announce the first edition of the Microsoft BizTalk Server 2006 R2 Hyper-V Guide.


 


The Microsoft BizTalk Server 2006 R2 Hyper-V Guide is the third installment in a series of guides intended to provide easily accessible, hands-on guidance to our customer and partner community. This 145 page guide is available on MSDN, TechNet and as a separate DOCX, PDF or CHM download alongside the already available Microsoft BizTalk Server Operations Guide”  andMicrosoft BizTalk Server Performance Optimizations Guide


 


The guide provides relevant information to IT professionals to enable them to make educated decisions about the advantages and tradeoffs of using Windows Server 2008 Hyper-V to virtualize BizTalk Server environments. This guidance was derived from a 6 week performance lab conducted by the BizTalk Customer Advisory Team and Premier Field Engineering.  It will be refreshed for future versions of Microsoft BizTalk Server.


 


The key sections of the guide are:



  • Getting Started: provides conceptual information about Hyper-V, the virtualization technology introduced with Windows Server 2008, and an introduction to the Hyper-V architecture.

  • Deploying BizTalk Server on Hyper-V: describes the steps that were followed to set up the lab environment, which was used to compare the performance of a BizTalk Server solution running on Hyper-V environment to the same BizTalk Server solution running on comparable physical hardware.

  • Evaluating BizTalk Server Performance on Hyper-V: details the important considerations when measuring the performance of a BizTalk Server solution running on a Hyper-V virtualized environment.

  • Testing BizTalk Server Performance on Hyper-V: provides detailed results of four distinct testing scenarios that compare the performance of a BizTalk Server solution running on Hyper-V environment to the same BizTalk Server solution running on comparable physical hardware.

 


The target audience for this guide is Microsoft field, partner organizations, and customers who plan, deploy, and maintain BizTalk Server installations.


 


 


Full MSDN URL:     http://msdn.microsoft.com/en-us/library/cc768518.aspx

Full TechNet URL: http://technet.microsoft.com/en-us/library/cc768518.aspx    

Adding a new BizTalk Server Isolated Host needs a new Application Pool

Recently, we tried creating a secondary Isolated Host to separate receive and send processing for WCF. After reconfiguring the servers, we were faced with the “Receive location for address ” not found. (The BizTalk receive location may be disabled)”. Puzzled, we looked around for configuration issues. After some reading (http://technet.microsoft.com/en-us/library/aa559328.aspx), we remembered a new isolated host needs a new application pool. Adding another application pool to the mix the the new host fixed our trouble…

Jotted down for posterity 😉

Parsing Timecard from Word to XML

We had to take a timecard that was in MS Word 2003 format and create XML. An example of the word document is as follows.

The red box indicates text that was hidden in the document which I have made visible for clarity. The program reads through the document, extracts text from the fields we are interested in and then converts it to XML as follows:

<?xml version="1.0" encoding="utf-8"?> <ns0:file xmlns:ns0="http://schemas.stottis.com/thomson/3e/proforma/detail"> <record> <date>Jun 04 08</date> <narrative>Time Entry Narrative</narrative> <hours>5.00</hours> <timecard>3330372</timecard> </record> <record> <date>Jun 05 08</date> <narrative>Time Entry Narrative</narrative> <hours>2.00</hours> <timecard>3330371</timecard> </record> <record> <date>Jun 07 08</date> <narrative>Time Entry Narrative</narrative> <hours>5.00</hours> <timecard>3330370</timecard> </record> <record> <date>Jun 08 08</date> <narrative>Time Entry Narrative</narrative> <hours>3.00</hours> <timecard>3330369</timecard> </record> </ns0:file>

Here it is: the answer to all SOA and ESB questions….

[[ OK, I am writing this with tongue-in-cheek, which for anyone not familiar with that term, means “not really serious”!!! ]]

Funny thing happened today on the ESB project that I’m on….

After a day of hard work, solving some really tough technical challenges (the kind that makes your head throb, your eyes glaze over and make you think that becoming a carpenter could be a smart career move), I leaned back in my chair and surveyed the landscape (my desk). A piece of tattered paper caught my eye, the piece of paper the team had been using all day long to describe concepts, flow and “you are here” type information to ourselves. But, I saw it as if I was seeing it for the first time, and I was struck by what it had evolved into. Then I noticed that it was actually just the latest diagram in a series (we had no white board in that room). The first one was crisp, and not marked up, and on a relatively small piece of paper (could have been a cocktail napkin). All the others in the series (there were three more leading up to the final one) were progressively more marked-up, scribbled on, and tattered, and all on A4 size paper. So, I though I would share with you that vital piece of paper that some very bright people spent the day with. And, I’m not sure what happened to that bottom-left corner.

This my friends, is actually the architecture of a very cool world-class, country-scale ESB. It solves ALL problems associated with ESB and SOA architecture. It is a comprehensive solution that also even layers on end-to-end governance. We have intermediaries, bus services, generic on-ramps, SLA enforcement, itinerary repository… we have it all. In fact, if you have ANY questions about how all these things fit together, it’s probably clearly explained in this diagram. This, in all its blazing glory, is real-world goodness I just had to share, for the benefit of the community.

Although this could be confidential in nature, I felt I was pretty safe posting it on my blog as the intellectual property is pretty much naturally protected. To the untrained eye, this looks like, well, I’m not sure what it would look like, probably just an unintelligible mess. But to us, it’s pure gold, and the roadmap to success. Enjoy!!!

And no, I will NOT do a webcast to explain it. Tomorrow is a whole new day, and, who knows, maybe even worthy of a fresh piece of paper. This, however, is perhaps one of the finest pieces of “software architecture art” in existance 🙂

Technorati Tags: esb,soa,biztalk,architecture

MsgBoxViewer (MBV) Q & A

MsgBoxViewer (MBV) Q & A

 


Q: What is the purpose of MBV ?
A: The goal of MBV is to retrieve the maximum amount of information from a BizTalk System in one click in the minimum amount of time, identify all possible issues which could be critical or need attention, and present them in a user friendly format.



Q: How is MBV different compared to other known tools like BPA or BizTalk Server 2006 Documenter ?
A: While BizTalk Documenter generates a nicely formatted overview document of a BizTalk Group, BPA and MBV are health check tools and generate reports of a BizTalk System that are complementary to each other.


   – BPA was developed by the PG for BizTalk 2K6 and reports two levels of warnings. BPA 1.1 was updated to contain some MBV rules but also contains a lot of additional rules relating to  BAM and the Rules Engine for example.
     BPA also provides detailed documentation on its own warnings.


   – MBV was developed by a Microsoft Support Engineer in collaboration with BizTalk Support and Field teams and contains some BPA rules but also adds additional ones.
     MBV reports two levels of warnings with corresponding links to MS KB articles, MS BizTalk Documentation, or BizTalk Whitepapers (but does not provide its own documentation of its warnings).
     MBV also retrieves detailed collapsible reports of information related to a BizTalk Group including its settings and its complete topology organized by servers, so MBV is more a mix of BPA and BizTalk Documenter.



Q: Which version of BizTalk Server can be analyzed with MBV ?
A: All versions and their SPs since BizTalk 2004 can be analyzed.



Q: Is MBV safe to run on a Production System ?
A: Yes, firstly, MBV does not need to be “installed” and secondly it uses only read SQL, WMI or VBS queries targeting the MsgBox, DTA, or Mgmt DB and uses WMI or VBS scripts to get info about the topology.
   The queries it contains were tested to not generate performance issues and locks



Q: Can MVB generate locks, deadlocks or Orphaned SPIDs on BizTalk DBs that it queries?
A: No, MBV uses only one thread for all its queries and does NOT use MSDTC transactions.
    If you have other health check tools querying your BizTalk DBs and running at same time as MBV I cannot predict the results.
    It is preferable to avoid using other Admin BizTalk tools when you use MBV.



Q: When I execute MBV, I get a “SecurityException” error
A: This error is raised when you start MBV from a remote location. You need to copy MBV locally and start it from a local folder 



Q: Is MBV able to analyze large BizTalk configurations involving multiple servers and clusters ?
A: Yes, MBV is able to identify and include in the topology report all servers and Cluster nodes involved in the BizTalk group


Q: Is MBV able to analyze multiple MsgBox DB configurations ?
A: Yes, MBV will identify and analyze all MsgBoxes in the BizTalk group



Q: Where do the SQL queries executed by MBV come from ?
A: Some of them come directly from BizTalk Architect Lee Graber’s “Advanced Queries” document. Other queries come from Microsoft BizTalk Support teams or known SQL & BizTalk Troubleshooting documents.



Q: Why do you have both a Console and a GUI version ?
A: The GUI tool interactively displays information collected in listviews and allows you to easily select which queries to execute. The Console version can immediately issue a Collect and can be launched by any type of parent application (like “PssDiagForBiztalk”) 



Q: Why is MBV updated so frequently ?
A: To include ASAP additional check rules & warnings based on issues met by Microsoft BizTalk Support and Field teams.
 


Q: Why does MBV not use .NET 2.0 ?
A: In order to be executable on BizTalk 2004 Servers which might only have .NET 1.1 installed



Q: Why does MBV have some Queries Listviews in Bold ?
A: The  list(s) in bold contains queries which will always be executed and are considered so as importanrt and the other lists offers you the option to choose the queries to execute.



Q: What is the recommended choice of queries to execute?
A: I recommend to  select ALL queries the first time in order to collect the maximum number of reports including a fully detailed Topology Report and to have maximum checks done so all possible warnings are raised in the Summary Report. You can then unselect some queries depending of the type of output you want or to have a faster collection time



Q: Why do I sometimes see timeout errors in the HTML or status log files ?
A: Some queries like Fragmentation or MSG Count in DTA db can take a long time to execute and so the default Query Timeout of 30 seconds will expire.
   In this case, the tool will report a timeout error but will continue to execute other queries



Q: Can we stop a pending Collect if it is taking too long to finish ?
A: Yes, MBV  offers this feature but the Collect will be stoped only after the pending query finish to execute or timeout.You can not stop a pending query.


 


Q: How does MBV manage the errors met by some queries ?
A: Each error generated by a query is caught in order that it cannot prevent the next queries from executing



Q: why do I have Access Denied Errors executing some queries ?
A: These Access Denied errors are caused mainly by missing SQL, WMI, or REGISTRY access permissions for the user running MBV.
Recommendation is to be a BizTalk Admin and SYSADMIN on the SQL Server(s) hosting the BizTalk Dbs but one of the assumptions that we often make with MBV is that the user running MBV will have complete access rights to both BizTalk Server and SQL Server but it is not always the case.

Q: How can I reduce the total duration of the Collect ?
A: Either by reducing the number of queries to execute, or by reducing the query timeout if you have too many time consuming queries 



Q: What is the real purpose of the status log file ?
A: This file will report ALL activity generated by MBV with the time stamp of each query and report generated. This file is also very interesting for the author to identify possible bugs



Q: What is the real purpose of the History log file ?
A: This file will contain in plain text ALL Collect outputs made since the last purge. This file can be purged in the GUI.



Q: Can the Query or Connection Timeout be changed ?
A: Yes, both the GUI and Console versions can have these timeouts changed



Q: Can I have the MsgBox or DTA db Tables size reports ordered by table name ?
A: Yes, you have the possibility to change the order type in the “Options” tab or by using a parameter in the Console version



Q: Can I know the collect duration from a HTML report ?
A: Yes the first line of the summary Report includes this information



Q: Sometimes MBV appears to not be responding, how d I know what it is doing ?
A: The status bar shows the pending query and the tool use two different threads for the GUI and the Collect engine so even during a long query the tool window is always visible and responsive.



Q: What is the role of the different tabs in the second half of the Gui tool ?
A: The tabs contain listviews of some important reports including the Summary Report and these listviews are populated during the Collect.
   You can navigate into these lists (except the Summary Report one) even if the collect is still pending !



Q: How can I have more know info about the queries to execute ?
A: For each query to be executed, the GUI version displays in its list the target and scope of the query, its caption, and its type (SQL, WMI, VBS, CUSTOM., etc…)



Q: What means really this entry in the Header Report ? : “This BizTalk Server is NOT a member of the targeted BizTalk group”
A: It means that the BizTalk server running MBV does not belong to the BizTalk group referenced by Mgmt DB server and name you have specified.
   By default when running on a BizTalk Server, MBV automatically tries to get the targeted Mgmt DB server and name from the registry



Q: Can I run MBV on a server which is not a BizTalk one ?
A: Yes, MBV can be run on any server that has .NET 1.1 installed no matter what it is. E.g. Win 2K, Win 2K3, Win XP or Vista.
If it is not executed on a BizTalk Server, you will have to specify the Mgmt DB Location and Name.
However, in this case, the query to collect local perfmon counters will not be executed and the ability to detect throttling will be missing! 



Q: Is it possible to develop some solutions analyzing automatically the MBV output file ?
A: Yes, MBV also allows the option to generate output in an XML file containing its own schema (in GUI version)
    Ideas of possible solutions to develop around XML MBV file :
    – Custom  Summary Report analyzer
    – Tool to build a Visio diagram from the Topology Report
    – Tool to identify differences between several MBV XMLs
    – …



Q: What is the SQL Query used behind the “Running SPIDs” or “Locked SPIDS” queries  ?
A: These two queries use a SP_who3.SQL script based on sp_who2 but updated for BizTalk  



Q: Why some Queries can execute only on SQL 2K5 ?
A: Some queries like “Most CPU SQL Queries” are based on SQL 2K5 relational rowsets through dynamic management views (DMVs).



Q: Can MBV run periodically and automatically some Collect ?
A: Yes, you have an option in the GUI version which allows you to configure a periodic interval in minutes to collect
   As soon you check the option “Collect Each”, a timer is started and will trigger the collect after the interval configured.
However best way to achieve that is to schedule  the start of the console version of MBV (BTSDBCOLLECT.EXE)



Q: Can I have access to previous HTML files generated ?
A: Yes, each Collect generates an HTML file in the specified output folder containing in its title the MBV version + current Date & Time to uniquely identify it.



Q: How is built the “Msg Flows” Report ?
A: This Flow report is built from the Subscription table but also from some orchestration port bindings tables.
    This report cannot show all possible flows!



Q: Is MBV able to retrieve BizTalk errors in the Application journal
A: Yes, MBV provide an optional query to to collect  error events of Biztalk sources



Q: When I open HTML file in IE on some machines, I can not expand the reports
A: MBV HTML files use scripting and IE can be configured to prevent scripting from executing for security reasons


Q: Can MBV send its warnings in eventlog only ?
A: Yes, MBV provides an option to send automatically its warnings in the application eventlog and provides also some options to prevent generating the HTML, Status, and History files.