You may already know that I usually use the series A fish out of water when I want to write something that goes a little bit off-topic on my main blog topic: Enterprise Integration. This time is not an Enterprise Integration topic but is somehow related to it since we are going to learn a possible way to try or troubleshoot SMTP issues that can be used as a channel on our integration solutions.
While working with BizTalk Server, sometimes I had the need to send email notifications or send messages thru email. In some cases, we use Office365 accounts to authenticate on the SMTP Server and send emails thru that account, but in some cases, we may have an internal SMTP Server with anonymous authentication that doesn’t require any account to send emails or a valid email to specify on the from. Of course, normally, that doesn’t mean that it is an “open bar” where everyone can send emails from and to anywhere they want. You should find some limitations thru other types of setting you can perform on the SMTP server or in the network layer like:
- Only specific machines can access and send emails thru that SMTP Server.
- You can only use a specific sender, or you can only send emails internally to the organization.
- and so on.
To try to see if everything is set up correctly before you use BizTalk Server, for example, or even thru troubleshooting errors, you can try to send an email thru the Command Prompt. To do that, you need to:
- Open Command Prompt using Start > Command Prompt or via Run > cmd
- You then need to do a telnet to the mail server by typing telnet <domain> <port> (usually, it is 25) and then pressing Enter.
- Once connected, we must initiate the mail-sending process by typing helo
- The server will reply with 250 and Hello if successful.
- We now need to specify the sending mail address by typing mail from:<email address> and then pressing Enter.
- The server will reply with 250 if it is a valid sender.
- After that, we specify the recipients by typing RCPT TO:<email address> and then pressing Enter.
- The server will reply with 250 if it is a valid receiver.
Most of the time, if errors exist, you will find them in this first part of the process.
Now to actually send an email, you need to:
- Type data and press ENTER to begin the email content.
- We first need to set the subject of the email by typing Subject:<Your Subject> and then pressing Enter twice.
- Now start typing the message content of your email.
- To finish and close the message, do the following sequence
- Press Enter
- Type . (Period Key)
- and press Enter again.
You may then receive a server response saying, for example, that the mail was queued for delivery.
- Type Quit and press Enter to exit telnet.
Now you just need to verify if your email was received.
With BizTalk Server 2009, setting up integration with Team Foundation Server (TFS) has become much simpler. While setting up continuous integration, automated unit tests, and msi packaging was possible before BizTalk 2009 it was a huge pain.
Below I will walk through the steps to set this up with BizTalk 2009. I was able to get this up and running in about 30 hours including the time to create the Virtual Machine (that was 15 hours). It took 47 build attempts in TFS before all the bugs were worked out in the process. While I cut some corners for the demo it would not take much more time to develop a true production ready solution.
We can start by taking a look at the Virtual Machine setup:
- Windows 2008 SP1
- TFS 2008 SP1 with Build Server installed
- SQL 2008 with all optional components installed
- Visual Studios 2008 SP1
- BizTalk Server 2009 with MSBuild tools installed
As you read though the steps below keep in mind I have about 10 hours of experience with MS Build and TFS 2008. This was my very first time setting up automated unit tests and continuous integration with BizTalk. This is just one approach for demo purposes. In real life, for example, all these systems would not be on the same server. This would surely make the process harder.
At a high level, this is what is happening:
Update to a file is checked in -> Build is kicked off -> Build completed with no errors -> Unit Tests are ran -> (Verification – Not Shown) -> MSI is created
Keys Pain Points:
- Setting up TFS 2008 with SQL 2008 is many times more complex that you would think. Make sure you Google this before starting.
- Remember user permissions. This will affect your share permissions, other folders, and the ability to run scripts. For example, to run the Create MSI Process below the user running the Build Agent will need to be a BizTalk Admin.
- Relative and absolute file paths are a killer. I spent a lot of time finding temp locations and getting relative paths to work. Looking at the Build Log and MSBuild Targets was a huge help.
- Keep in mind TFS will use the version of code checked into TFS. If you update a file or the build project make sure you check it in or the new settings will be ran.
Download the Solution Code
Setting Up Continuous Integration, Unit Tests, and MSI Creation in BizTalk 2009 Sample Code
Setting Up Unit Tests for use in Continuous Integration
Step 1: Setup a Unit Test project following the help guide instructions at http://msdn.microsoft.com/en-us/library/dd224279.aspx
Step 2: Create a Test List in the .vsmdi that was added to the solution when the Unit Test project was created. Right click on the Lists of Tests. Creating the new list named RunAllUnitTests is shows below.
Step 3: Add the test methods from Step 1 to the new test list. Drag and drop the test into the test list. This is shown below.
Pointers: The hardest part of setting up the unit tests is getting the file paths correct for Schema and Map testing. I finally got tired of trying to figure it out and hard coded the paths to known local files. This is not the right way to do it.
Setting Up Continuous Integration
Step 1: Add Solution to Source Control in its own folder tree. In this case it is called CIDemo as shown below.
Step 2: Create a new Build Definition inside Team Explorer.
Step 3: Give the Build Definition a name. In this case it is called CIDemoBuildDefinition.
Step 4: Set the Workspace to the CIDemo solution folder created in Step 1. This is shown below.
Step 5: Go to the Project File section of the wizard. Select Create on the Project File screen to make a new Build Project. A new wizard will open.
Step 6: Select the CIDemo solution. This will be the solution that the build project will build.
Step 7: Select the build type. In this case it is Release.
Step 8: Since we already created the Unit Tests and a Test List select the RunAllUnitTests test list to have the unit tests ran when a build is performed. This is shown below. This can always be updated in the build project later on if the Unit Tests are not ready. Click Finish to end this wizard.
Step 9: Back on the main wizard, leave the Retention Policy items unchanged.
Step 10: Under Build Defaults, select New to create a new Build Agent. Name the build agent and set the computer name. In this case the name is CIDemoBuildAgent and the computer name is Win2008Ent-Base as seem below.
Step 11: Set the Share location for the builds to be copied to, also known as the Drop Location. A local share was created called Builds. To ensure no permission problems Everyone was added to this share. This is not what should be done in real life. Most likely this would be on another server.
Step 12: Under Trigger, select the Build each chick-in radio button. This will create a new build with each check in. Click OK to create the Build Definition.
Step 13: Test the process. Check in a file.
Creating A BizTalk MSI
The process used here to build an MSI package first installs the BizTalk Assemblies and Binding file to a local BizTalk Server. Then it exports out the MSI Package. While other approaches can be used that do not require a local BizTalk instance, this approach would allow for additional BizUnit style Unit Test (or Build Verification tests) to be ran against deployed code.
Step 1: Modify the CreateFullandPartialMSI.bat sample file in the CreateApp folder under Application Deployment in the BizTalk 2009 SDK. This file is called BuildMSI.bat in the Helper folder in the solution. Changes made to the file include changing paths, dll names, and application names. Make sure the order of the dlls is in the correct deploy order. i.e. Schemas before Maps.
Step 2: Modify the Build Project created in Step 5 above. This file is a MSBuild file that controls the build and tests ran against the build. At the end of the file right before the closing </Project> tag add the following:
<!– This Target created a directory for the binding files and Copies them and the build bat file to the temp directory. –>
<MakeDir Directories="$(BinariesRoot)/Release/Bindings" ></MakeDir>
<Copy SourceFiles="$(SolutionRoot)/Bindings/CIDemo_Bindings_Dev.xml" DestinationFiles="$(BinariesRoot)/Release/Bindings/CIDemo_Bindings_Dev.xml"></Copy>
<Copy SourceFiles="$(SolutionRoot)/Helper/BuildMSI.bat" DestinationFiles="$(BinariesRoot)/BuildMSI.bat"></Copy>
<!– This Target runs the build bat file, copied the completed MSI, and deletes the bat file from the file share. –>
<Exec Command="$(BinariesRoot)/BuildMSI.bat" WorkingDirectory="$(BinariesRoot)" ></Exec>
<Copy SourceFiles="$(BinariesRoot)/CIDemo.msi" DestinationFiles="$(DropLocation)/$(BuildNumber)/CIDemo.msi"></Copy>
This code will copy the binding files, the bat file used to build the MSI, and do some clean up. This can be customized as needed and the possibilities are almost endless. Make sure the updated file is checked into TFS.
Step 3: Ensure the user account running the build agent is a member of the BizTalk Admin Group. This use can be found inside TFS by starting a build and viewing the properties as seen below. This account is set up when you install TFS.
Step 4: Watch for output in the folder share when a file is checked in or a build manually started.
This outlines at a high level the process to create automated unit tests, set up continuous integration, and create a BizTalk MSI package. I hope to put this together into a video shortly. Until then, best of luck.
This is a follow up to my post yesterday. I was setting up some automated unit tests for BizTalk 2009 in Release mode. I received the following errors.
CIDemo.Schemas.InputSchema does not contain a definition for ‘ValidateInstance’ and no extension method ‘ValidateInstance’ accepting a first argument of type ‘CIDemo.Schemas.InputSchema’ could be found (are you missing a using directive or an assembly reference?)
CIDemo.Maps.Input_To_Output does not contain a definition for ‘TestMap’ and no extension method ‘TestMap’ accepting a first argument of type ‘CIDemo.Maps.Input_To_Output’ could be found (are you missing a using directive or an assembly reference?)
CIDemo.Maps.Input_To_Output does not contain a definition for ‘ValidateOutput’ and no extension method ‘ValidateOutput’ accepting a first argument of type ‘CIDemo.Maps.Input_To_Output’ could be found (are you missing a using directive or an assembly reference?)
This is because the value under Unit Testing: Enable Unit Testing are Configuration Specific as seen below. The default is False. So moving to Release mode reset the values.
I also noticed the Application Name is Configuration specific as well.
Just something to keep in mind when changing configurations in BizTalk 2009.
Today I was setting up Unit Tests for BizTalk 2009 Schema and Maps. I was using the BizTalk Server 2009 help guide example as a reference.
I ran into the following error message:
<Class Name> does not contain a definition for ‘ValidateInstance’ and no extension method ‘ValidateInstance’ accepting a first argument of type <Class Name> could be found (are you missing a using directive or an assembly reference?
This looked like this inside Visual Studios 2008:
This was caused by the referenced Schema project not being marked to Enable Unit Testing. This setting can be found under the Product Properties.
Just simply change this to true, rebuild the Schema and Maps Projects, and the Unit Test methods should now be able to find the ValidateInstance or other missing methods method.
When Unit Testing BizTalk 2009 Map projects, the ValidateOutput method may not be found. While I did not have problems with creating mapping unit test the solution would be the same.
Recently I have been working a lot with BizUnit 2.0 (Available on GotDotNet). This framework provides an easy and customizable way to unit test your end to end BizTalk solutions. I’ll not get into details about BizUnit right now since this post is focused on Visual Studios 2005 Code Snippets.
Young Jun has developed a set of code snippets for BizUnit 2.0 for use with Visual Studios 2005. Although not all the BizUnit tasks are included, it is a huge time saver when building unit test files.
The only problem I had was how to install the snippets. I found it a little tricky so here are the steps I went through.
1. Download the code snippets.
2. Unzip them to a folder, in my case C:\BizUnit
3. Open Visual Studios and locate the Code Snippets Manager under Tools.
4. If you do not see it listed, you will need to go to Customize under Tools, look under Commands, Tools, and move the command to the Tools menu.
5. Change the Language to XML
6. Select the Add button and select the folder with your snippet (in my case C:\BizUnit)
7. Click OK.
8. You are all set. Now when you open an Xml document inside Visual Studios you will see the BizTalk Snippets available
More to come on BizUnit in later posts.
A few days ago Microsoft released a Load Generation tool to help simulate load for testing Biztalk 2004 solutions. You can download this testing tool here.
It seems that the installation package defaults the installation path for the testing tool to something other then your C drive. In one case, my installation went to my E:\ drive and in the other two cases it went to my D:\ drive. Just make sure during the installation you note (or change if you want to) the installation location.
Also, it appears that if you do not have MSMQ installed you will get an error saying MSMQTransmitter.dll did not register. I said Ignore and it seemed to install correctly. I tried this on the computer with MSMQ and did not have this problem.
I have not done anything more then install the program and briefly look at the documentation. It seems quite powerful, but Larry Beck’s tool is much simpler and I didn’t have any problems installing it.
Thanks to Bryan Corazza for the release information.
Larry Beck from Avanade has released a Biztalk Throughput and Capacity Testing Tool for testing Biztalk Performance.
This tool will automatically generate files for you to submit them into Biztalk. In addition, it will track server parameters in a nice graph. It’s configurable through a UI.
You can get more information on his blog or on the GotDotNet workspace.
Who is Larry Beck? He is a Connected System Evangelist and fellow Texan who works for Avanade. Avanade is a joint venture between Accenture and Microsoft. If you went to TechEd, you probably saw there video in the Key Note.
Look for more good things to come from him blog!