.gif)
by community-syndication | Aug 13, 2008 | BizTalk Community Blogs via Syndication
I've been working in a fairly big BizTalk project since early this year, and wanted to share my experience working with Team Foundation together with BizTalk projects. The solution became more complex than originally anticipated. So hopefully you can learn from this, and maybe reuse some of the code and scripts I'll share in this series of articles.
But before we get into the details, let me fill you in on the requirements and challenges…
Easy and automated Deployment
- We have five types of environments: Development (several), Build, Test, Acceptance and Production. We have about 30 BizTalk applications with a total of just about 300 ports on each environment.
- We have normalized our BizTalk projects as much as possible, to reduce impact on redeployment, which left us with just over 200 Visual Studio projects. The amount of projects, together with its dependencies, would make deployment quite challenging.
- Each developer must be able to build and deploy there solutions, independently from other developers.
- Due to the complexity of the environments, the deployment process has to be rigid or preferably automated.
Looking at Microsoft Team Foundation (from a far distance), it seems to be the solution to our problems. However, on a closer look there where many obstacles to overcome, before we'd get it to work.
Team Foundation Server
Team Foundation is a server product that integrates with products like MS Excel, MS Project and of course Visual Studio. This is accomplished through Team Explorer which adds functionality such as WorkItem Tracking and Source Control to these products.
Team Foundation also provide the functionality for Team Build, through which Build Managers can compile the projects, run associated unit tests, perform code analysis, release builds on a file server, and publish build reports.
.gif)
The basic practice is this:
- Create a Visual Studio Solution with your projects, and check them in to the source control.
- Create a Build, and select your solution, along with the Build server to use.
- You may optionally choose triggers for when you want the Build to start. Useful for nightly builds etc.(only for Team Explorer 2008)
- Run the Build.
When running the Build, a Team Build Service on the selected Build server will get the latest code, label it, and build it. This is all done according the tfsbuild script that was created for you when you created the Build.
However…
Everything works fine unless, of course, you're trying to build BizTalk projects… As it turns out, Team Build is very similar to MS Build, and normal Visual Studio projects like C# projects, are just MS Build scripts. BizTalk projects, however, are not, and are therefor ignored by the build engine.
We didn't want to invest in Team Foundation 2005, but since we can only use Visual Studio 2005(VS 2008 does not support BizTalk projects yet), we had to use Team Explorer 2005, which gives less functionality. This left the Build Manager to have both Team Explorer 2005 and 2006.
Furthermore, there are no built-in support to deploy BizTalk projects.
Solution
To meet our requirements and utilize the functionality of Team Foundation, we had to start out by setting some design rules.
Since each developer must be able to build and deploy there solutions, independently from other developers, we had to group our projects in groups without dependencies to other groups. This was implemented by grouping the BizTalk projects in to ILS solutions. ILS is a term sometimes used in earlier BizTalk version and stands for Integration Service Layer. But in this case, it's better read as Isolated Service Layer, since there can be no references to projects in other ILS solutions.
Each ILS solution also included binding files for each BizTalk Application and environment.
A BizTalk project has by default two build configurations, Development and Deployment. We used these configurations to set the Target Environment (Development == (local), Deployment == Build server). Later on, when deploying the project through Team Build, we used the Deployment configuration. The accrual compiling of each BizTalk project was done by DevEnv.bat rather than MSBuild. By using the /Deploy flag, we where actually able to build, deploy and create the BizTalk Application in one line of code.
"C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\devenv" "MySolution.sln" /Deploy "Deployment"
We ended up creating a handful custom build tasks, which we could use in our build script to:
- Undeploy ILS projects, ports and artifacts
- Build and deploy all ILS projects
- Import Binding files to Build environment
- Generate MSI packages for deployment in other environments with only the deployed artifacts and binding files.
This all led us to be able to be able to, through Team Build, build all ILS projects, deploy them to the build environment, label all sources and open up for an easy deployment for the other environments.
In the next article I promise to get into more details, of the solution.
by community-syndication | Aug 13, 2008 | BizTalk Community Blogs via Syndication
By popular demand, I have put the ability to count in the EDI reporting. This allows you to count up to three different objects in the data (via an XPATH statement). Here is the pipeline component:
And here is the updated view:
by community-syndication | Aug 13, 2008 | BizTalk Community Blogs via Syndication
Firstly, if you haven’t already seen it, have a look at the post describing the EnableSafeTyping binding property – http://blogs.msdn.com/adapters/archive/2007/11/24/the-enablesafetyping-binding-property.aspx.
Since the Adapter Pack V1 released, we have had requests from users, wherein, they want to expose the Date (DATS) and Time (TIMS) data types as a .NET DateTime (strong typing), but yet want the adapter to be able to handle common SAP values such as 00000000 for DATS (which in SAP would mean a MINVALUE or NULL for the Date field). Besides, we already know that SAP doesn’t validate the data in DATS, TIMS and NUMC fields, so you might end up receiving “invalid” data from a SAP RFC, but you yet want the strong typing in your client code.
As a result, we’ve added a new binding property named “DataTypesBehavior” in the SAP adapter. This property is actually a “complex” property, containing 12 sub-properties. I’ll first list the names of the sub-properties.
- DateTimeMaxToDats
- DateTimeMinToDats
- DateTimeMaxToTims
- DateTimeMinToTims
- DatsMaxToDateTime
- DatsMinToDateTime
- TimsMaxToDateTime
- DateTimeNullToDats
- DateTimeNullToTims
- InvalidDatsToDateTime
- InvalidNumcToInt
- InvalidTimsToDateTime
- EmptyDatsToDateTime
I’ll explain just a few of the above; you should then be able to figure out what the rest mean.
- DateTimeMaxToDats – this property controls the behavior when converting DateTimeMax (the .NET DateTime.MaxValue value) “To” Dats (the SAP DATS value). The .NET DateTime.MaxValue can only be present in the request message. The adapter needs to convert this to a DATS value when sending it to SAP. Therefore, this property controls the behavior of how the adapter parses the request message.
- DatsMaxToDateTime – this property controls the behavior when converting DatsMax (the SAP DATS MaxValue, which is 99999999) “To” DateTime. DATS values can only be sent from SAP right? And, only a client consuming the adapter can understand .NET DateTime, right? Hence, this property controls the behavior of the adapter, when it receives the DATS value from SAP (as the response to a RFC invocation), and when the adapter is converting this value to XML in order to give the response to the client.
Additional information for each property, including details on the values which need to be specified for each property, can be obtained by peeking through the attached .txt file – it contains the comments present in the source code of the adapter for these individual properties.
NOTE – this binding property is present in the SAP Adapter in the Adapter Pack v2. For the Adapter Pack v1, there is a hotfix available (KB # 954539) – your Microsoft support contact should be able to obtain it for you.
by community-syndication | Aug 12, 2008 | BizTalk Community Blogs via Syndication
On this BizTalk Glossary Page it states that a zombie ressurector is: An orchestration that listens at the MessageBox database for zombies and resurrects those that meet certain criteria.
The previous definition of a zombie is: A message that was not processed by an orchestration. It is not in the Suspended queue or anywhere else where it will be retried.
If there is an orchestration that is looking for ‘zombies’ – then they were never zombies were they?
Once you have answered that, then you can answer the question of which came first, the chicken or the egg?
by community-syndication | Aug 12, 2008 | BizTalk Community Blogs via Syndication
I have needed for some SQL jobs to run every second. I went into the Jobs and started to configure the job, unfortunately, the lowest value I can choose is minute:
So I found that in the script, the underlying values for Occurs (@freq_subday_interval) every are as follow:
| Value |
Description (unit) |
| 1 |
At the specified time |
| 2 |
Seconds |
| 4 |
Minutes |
| 8 |
Hours |
So I scripted the schedule like this:
EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'EverySecond',
@enabled=1,
@freq_type=4,
@freq_interval=4,
@freq_subday_type=2,
@freq_subday_interval=1,
@freq_relative_interval=0,
@freq_recurrence_factor=0,
@active_start_date=20080812,
@active_end_date=99991231,
@active_start_time=0,
@active_end_time=235959
I got the following error:
Msg 14278, Level 16, State 1, Procedure sp_verify_schedule, Line 214
The schedule for this job is invalid (reason: The specified ‘@freq_subday_interval’ is invalid.).
Here is the logic in the sp_verify_schedule
IF ((@freq_subday_type <> 0x1) AND (@freq_subday_interval < 1))
OR
((@freq_subday_type = 0x2) AND (@freq_subday_interval < 10))
BEGIN
SELECT @reason = FORMATMESSAGE(14200, '@freq_subday_interval')
RAISERROR(14278, -1, -1, @reason)
RETURN(1) -- Failure
END
So there are a few things that can be done to get around it:
In the job use the following code:
EXEC stored_proc;
WAITFOR DELAY '00:00:01';
EXEC stored_proc; -- again
WAITFOR DELAY '00:00:01';
EXEC stored_proc; -- and again
Or by tracking through the various stored procedures (sp_add_jobschedule and sp_add_schedule) we see that the following table is where it ends up: msdb.dbo.sysschedules, so we can just go update the table after we scheduled the job.
UPDATE msdb.dbo.sysschedules
SET name = ISNULL(@schedule_name, name),
enabled = ISNULL(@enabled, enabled),
freq_type = ISNULL(@freq_type, freq_type),
freq_interval = ISNULL(@freq_interval, freq_interval),
freq_subday_type = ISNULL(@freq_subday_type, freq_subday_type),
freq_subday_interval = ISNULL(@freq_subday_interval, freq_subday_interval),
freq_relative_interval = ISNULL(@freq_relative_interval, freq_relative_interval),
freq_recurrence_factor = ISNULL(@freq_recurrence_factor, freq_recurrence_factor),
active_start_date = ISNULL(@active_start_date, active_start_date),
active_end_date = ISNULL(@active_end_date, active_end_date),
active_start_time = ISNULL(@active_start_time, active_start_time),
active_end_time = ISNULL(@active_end_time, active_end_time)
WHERE schedule_uid = @schedule_uid
I decided to modify the sp_verify_schedule
IF ((@freq_subday_type <> 0x1) AND (@freq_subday_interval < 1))
OR
((@freq_subday_type = 0x2) AND (@freq_subday_interval < 1))
BEGIN
SELECT @reason = FORMATMESSAGE(14200, '@freq_subday_interval')
RAISERROR(14278, -1, -1, @reason)
RETURN(1) -- Failure
END
Did I happen to mention the disclaimer on the right side of my blog?
by community-syndication | Aug 12, 2008 | BizTalk Community Blogs via Syndication
So now I can finish my installation of SQL Server 2008 RTM!
http://msdn.microsoft.com/en-us/vstudio/products/cc533447.aspx
by community-syndication | Aug 11, 2008 | BizTalk Community Blogs via Syndication
While modifying a file for a BizTalk process for a client we kept on getting an Error using message tracking in HAT looking like the following:
System.NullReferenceException: Object reference not set to an instance of an object.
at Elite.EAI.Orchestrations.Receive_Orchestration.segment8(StopConditions stopOn)
at Microsoft.XLANGs.Core.SegmentScheduler.RunASegment(Segment s, StopConditions stopCond, Exception& exp)
System.NullReferenceException
Scoped@
Receive_Orchestration.Trans_CobraScope
Receive_Orchestration.Receive_Orchestration
68223de9-f824-4fbb-868b-35dd6b61eb4b
The only thing that overcame this problem was changing the file name to something the BizTalk application recognized. In our case that was UPS for a UPS process. Adding a GUID also threw the same exception. Adding characters to the end didn’t throw the exception.
by community-syndication | Aug 11, 2008 | BizTalk Community Blogs via Syndication
By default, CLR is disabled (for security reasons of course), so if you think that writing C# code is easier to call stored procedures compared to actually writing the stored yourself, or otherwise want to extend what SQL Server can do out of the box, you need to enable it.
Here is how you do it:
sp_configure 'clr enabled', 1
go
reconfigure
go
by community-syndication | Aug 11, 2008 | BizTalk Community Blogs via Syndication
Anyone who has worked with the ESB Guidance Toolkit will tell you that the complex and troublesome installation of the toolkit is one of the most common reasons for the toolkit not getting wider coverage and usage. The next release of the toolkit…(read more)
by community-syndication | Aug 11, 2008 | BizTalk Community Blogs via Syndication
We’ll I’ve been back to work now for a few weeks. We had an amazing family holiday, driving through the US and Canada.
Click below to see the route we took (more or less)…
http://maps.google.com.au/maps?f=d&hl=en&geocode=10351655480452428531,40.374590,-105.499610%3B12810270970227554526,40.554450,-106.263990%3B3924502922590039050,44.348670,-103.451740%3B15866937145172212178,46.962090,-104.189040%3B5056530365340906547,51.195420,-102.451030%3B5297887049552016546,53.084010,-105.518510%3B3632703890947654852,54.496460,-103.832680%3B7943638280868292688,51.079120,-114.184360%3B3914372709131752317,52.811520,-118.044200%3B8898398290185831071,51.240140,-116.654310%3B13306813490729550546,49.338160,-123.139960%3B240439335949052098,48.500840,-123.385460%3B1069571150544135368,48.121630,-123.430810%3B822223714608053557,47.594150,-122.320730%3B17128118707005160925,43.291360,-123.353930%3B4638331477922862822,42.437886,-124.428667%3B10115102611937038347,38.558630,-122.820180%3B10042333566949135992,37.409250,-121.878950%3B12641313789675360573,35.101280,-120.548680&saddr=Denver,+CO,+USA&daddr=N+St+Vrain+Ave%2FUS-36+%4040.374590,+-105.499610+to:CO-125+%4040.554450,+-106.263990+to:I-90+E+%4044.348670,+-103.451740+to:MT-7+%4046.962090,+-104.189040+to:HWY-16%2FTrans+Canada+Hwy%2FYellowhead+Hwy+%4051.195420,+-102.451030+to:HWY-3+%4053.084010,+-105.518510+to:HWY-106+%4054.496460,+-103.832680+to:16+Ave+W%2FHWY-1+W%2FTrans+Canada+Hwy+W+%4051.079120,+-114.184360+to:HWY-93%2FIcefields+Pkwy%2FPromenade+des+Glaciers+%4052.811520,+-118.044200+to:HWY-1%2FTrans+Canada+Hwy+%4051.240140,+-116.654310+to:HWY-1+W%2FTrans+Canada+Hwy+W+%4049.338160,+-123.139960+to:HWY-17+S%2FPatricia+Bay+Hwy+%4048.500840,+-123.385460+to:Port+Angeles-Victoria+BC+Fry+%4048.121630,+-123.430810+to:I-5+S+%4047.594150,+-122.320730+to:I-5+S+%4043.291360,+-123.353930+to:US-101+%4042.437886,+-124.428667+to:US-101+S+%4038.558630,+-122.820180+to:I-680+S+%4037.409250,+-121.878950+to:36.004673,-121.530762+to:US-101+S+%4035.101280,+-120.548680+to:lax&mra=dpe&mrcr=0&mrsp=19&sz=7&via=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&sll=36.394757,-119.256592&sspn=4.164597,9.492188&ie=UTF8&ll=44.339565,-99.404297&spn=29.565191,75.9375&z=4
Anyway, since SQL Server 2008 RTM’d last week, I’ve been putting together a virtual machine image that I intend to use for a pre-conference session that I am delivering at Tech Ed, here in Australia.
Unfortunately, there is a bit of a snag in that, if you are going to use Business Intelligence Studio – there is a dependency on Visual Studio 2008 SP1, which won’t be released until later this week!
So, because of deadlines for this image, I will have to stick with RC0….
