BizTalk Benchmark Wizard – New release

by | Apr 23, 2010 | BizTalk Community Blogs via Syndication | 0 comments

The Benchmark Wizard was release earlier this year, and even though we got lots of good feedback we’ve also got requests for some changes.

If you haven’t yet heard about this tool, the purpose of it is to verify your BizTalk environment performs as expected. There are two different scenarios you can run, a Messaging and an Orchestration scenario. Each of the scenarios has been tested on various environments and configuration. The result of these tests has provided the tool with a set of KPI’s, which your test result will be benchmarked against. For more information about the Benchmark Wizard:

Benchmark your BizTalk Server (Part 1) – Overview

Benchmark your BizTalk Server (Part 2) – How to install

Benchmark your BizTalk Server (Part 3) – Drill down and analyse

The most common feedback is related to the fact that it was very difficult to meet the KPI’s. A reason for this is that the original tests where executed while global tracking was disabled. This was unfortunate as the DTA tracking has ~30% overhead. We have done a re-run of all the tests, and updated the KPI’s for the new version (shown at the bottom of this page).

Further more, there seams to be a question of how to interpret the result. What does “Succeeded” mean? Does it reflect the best possible result or good enough? To make this more clear, we’ve implemented the “stop light approach”, where if you’ve Succeeded you should be proud of yourself and make a blog post, while an Acceptable result is nothing to be ashamed of.     

 

 

There has also been some other fixes such as the resetting all the counters when you re-run the test, and fixing the CPU counters to show correct values.

Special thanks to Microsoft and Ewan Fairweather for letting me use their test lab!

Download BizTalk Benchmark Wizard from CodePlex

Scenario KPI’s: Messaging Single and Multi Message Box

# of Bts Srv

#CPU/Bts Srv

# SQL Srv

#CPU/SQL Srv

Msgs/Sec

Msgs/Sec
1 1 Quad (1) (1) 160 200
1 1 Quad 1 1 Quad 280 350
1 1 Quad 1 2 Quad 390 490
1 1 Quad 1 2 Quad 560 700
2 1 Quad 1 2 Quad 620 770
2 2 Quad 1 2 Quad 730 910
2 2 Quad 1 4 Quad 780 980

Scenario KPI’s: Orchestration Single Message Box

# of Bts Srv

#CPU/Bts Srv

# SQL Srv

#CPU/SQL Srv

Msgs/Sec

Msgs/Sec
1 1 Quad (1) (1) 110 140
1 1 Quad 1 1 Quad 170 210
1 1 Quad 1 2 Quad 190 240
1 1 Quad 1 2 Quad 220 270
2 1 Quad 1 2 Quad 230 290
2 2 Quad 1 2 Quad 260 320
2 2 Quad 1 4 Quad 300 370

 

Test environment:

Type Model CPU Type # of CPUs # of Cores/CPU Architecture RAM Local Disks OS Software
Database DL785 Intel Xeon 8 x 2.4 Ghz 4 x64 130 GB 2 x 72gb* Win2k8 SP2 EE 64bit SQL Server 2008 SP1
BTS Receive host R805 Intel Xeon 2 x 2.33 Ghz 4 x64 8 GB 2 x 72gb 10k SAS Win2k8 SP2 EE 64bit BizTalk Server 2009
BTS Send host R805 Intel Xeon 2 x 2.33 Ghz 4 x64 8 GB 2 x 72gb 10k SAS Win2k8 SP2 EE 64bit BizTalk Server 2009
Load server R805 Intel Xeon 2 x 2.33 Ghz 4 x64 8 GB 2 x 72gb 10k SAS Win2k8 SP2 EE 64bit BizTalk Benchmark Wizard
Back-end server R805 Intel Xeon 2 x 2.33 Ghz 4 x64 8 GB 2 x 72gb 10k SAS Win2k8 SP2 EE 64bit Indigo Service

* Storage: EMC Clarion CX-240 ( 5 solid state drives )

Configuration:

Submit a Comment