Corey Ladas has written an interesting paper titled Scrum-ban in which he describes how a Scrum team might introduce the lean practice of kanban. He goes on to describe an evolutionary process, which if taken far enough, replaces most of Scrum. Even for Read More……(read more)
I’ve just spent a great weekend in London with the parents, doing the tourist ‘thing’ (the photo above was taken from the London Eye); In September I’m off to New York for my 30th, then in November I’m off to Paris and Egypt – I think I’ll need Christmas off to recover!!
I’m really overwhelmed with […]
While setting up my SQL Server 2008 RTM image, I needed to create a demo using PerformancePoint.
After installing PerformancePoint (by the way, I had to point it to a SQL Server 2005 instance for it’s own databases, it didn’t like 2008) I went to connect to an Analysis Services 2008 data source. I got an error, something about data source not found. I wasn;t sure if this was going to be a problen with 2008 cubes in general.
After some investigation, I realised the problem was not with connecting to SQL Server, but rather PerfPoints own web service. In IE, I browsed to http://localhost/WebService/PmService.asmx. There was an error message “could not system.web.extension version=1.0.61025.0.“
A couple of articles talked about some workarounds,
http://nickbarclay.blogspot.com/2007/11/pps-data-source-connection-problems.html
http://hassansyed.blogspot.com/2008/02/ms-performance-point-connection-issue.html
But they looked like a bit too much hard work. So I tried installing ASP.NET Ajax 1.0 from
http://www.microsoft.com/downloads/details.aspx?FamilyID=ca9d90fa-e8c9-42e3-aa19-08e2c027f5d6&displaylang=en
and everything is sweet now. PerformancePoint Dashboards working fine connecting to SQL Server 2008 cubes!
Given the vast amount of information transported between systems in today’s enterprise environments, a lot of which is considered either private (addresses, social security numbers, credit card information, mortgage and pension plans, etc) or of high business value, information security is key.
From the release of WCF, it has brought us tremendous flexibility and allowed development teams to be almost completely agnostic of the way information is transported between point A to point B, focusing on adding business value instead.
Another great feature WCF brings us is runtime troubleshooting. In WCF, an administrator can simply enable logging to troubleshoot an environment which is in production.
Although a great feature for troubleshooting, Personal Identifiable Information (PII) you care about deeply while on the wire (using transport and message-level security to protect it) is available in plain text if logging is turned on at the server side.
WCF’s logging features currently filter out specific PII by default (only if the message is not malformed). However, this functionality is not exposed to the developer.
Consider the following scenario:
You could write your own logging subsystem and hook it in to your services. This takes a lot of work, is error prone, needs custom maintenance, documentation, etc.
My proposal for the scenario:
Let’s have a look at how the solution works:
Take a DataContract, e.g. PurchaseOrder and apply an attribute (PII) on each property which represents PII.
Next, add a PIIProtectionBehavior attribute to your service implementation class
You’re done. The PIIProtectionBehavior attribute is also an IServiceBehavior implementation which performs the introspection on the EndPoints.
The result when logging is turned on:
As you can see, ‘CreditCardnumber’ is no longer logged, it just displays ‘<– Removed –>’ .
I’ve attached the sample code, hope it’s useful!
Ofcourse, the normal disclaimers apply, the accompanied sample is for educational purposes only, it has not gone through rigorous testing, might contain bugs due to limited testing (me ;-)), etc. Please see the sourcecode for further remarks.
The London Underground announcers constantly use a phrase at terminii – “This train terminates here. All change please!!”. That phrasekinda resonates with me now cos in terms of career trains, my old one terminated yesterday and from today its all changed. I’m rather chuffed to say that I’ve joined Microsoft Consulting Services as senior consultant. […]
New to the web this week on the WF/WCF front:
Compared to other weeks in the past month, this has been a relatively quiet one on the publication front. There are a lot of really cool things being brought into the pipeline, especially for around the PDC timeframe.
Also, we have additional functionality coming to MSDN in the coming month. You may have already noticed that the MSDN servers are starting to become more community focused (including the new forums site and the MSDN bookmarks site and the green ‘Community Content’ sections that appears on the bottom of MSDN documentation). We’ve been working with the MSDN team to get new content for WF and WCF content to go onto the new service at launch; I’ll be interested in your feedback. The initial pages at creation is to place up about four or five initial pages that provide an inventory/collection of videos/screencasts and code samples that are out there. We are also thinking about creating a ‘Learning Roadmap’ to help those new to the technologies get a grasp on where to start and some help with understanding what’s out there today. But – as I said – thoughts/opinions/comments are welcome.
I apologize for the tardiness in getting this posting up. I typically work on these either Friday afternoon or Saturday morning, but it was a busy week in the office and I got drawn into writing my first Windows Mobile application over the weekend (a simple flash card program to help me learn the notes on the grand staff).
We’ve had a few users complaining about the throughput of the adapters (the Adapter Pack ones, and/or custom ones) coming to a standstill during normal operation within BizTalk. In this post, I’ll explain why that happens, and some workarounds.
Let’s say BizTalk has received 100 messages which it needs to send to the adapter on a Send Port. Assume that it queues 50 work items on 50 thread pool threads, with each work item containing 2 messages to be processed.
Now, on each of the above thread pool threads, the logic used is something similar to:
1. for (int i=0; i < numberOfMessages; i++)
2. {
3. IRequestChannel channel = CreateNewIRequestChannel();
4. channel.Open();
5. channel.BeginRequest(message[i], callbackFunction);
6. }
IRequestChannel above maps to an adapter channel – IOutboundHandler in the Adapter SDK.
Now, many of the LOB systems for which the adapters are written are do not have an asynchronous version of the LOB API or SDK, i.e., the LOB API is synchronous – blocks the current thread while the invocation occurs. However, as can be seen above, a call to BeginRequest() is made, which means that the caller wants the adapter channel implementation to return immediately. Therefore, in the Adapter SDK, the implementation is:
7. BeginRequest(Message message, Callback callbackFunction)
8. {
9. ThreadPool.QueueUserWorkItem(channel.Request, message, callbackFunction);
10. return immediately;
11. }
What the Adapter SDK does is, since it needs to return immediately, it queues an additional work item in the thread pool queue. The work item, when run, will call Request(), which is a synchronous call, which maps to IOutboundHandler::Execute() in the adapter, with the message as the parameter. Once the adapter finishes processing the message (synchronously) on the thread pool thread, the Adapter SDK takes care of invoking the callback function to notify BizTalk that the message has completed processing, and to hand the response message to BizTalk.
Also, one more piece of information – the channel.Open() call in line number 4 above, is handled by the Adapter SDK. When the Open() call reaches the Adapter SDK, it tries to obtain a free connection from the connection pool, or create a new connection to the LOB provided the maximum number of connections hasn’t been reached. If it is unsuccessful in both attempts, it blocks until a connection becomes available. Note – The adapters in the Adapter Pack expose a setting (typically named MaxConnectionsPerSystem, or MaxPoolSize, etc); the value of which is passed on to the Adapter SDK via a setting it exposes; custom adapters might expose something similar which the end user can tweak.
Given the two blocks of code above, and the behavior of the Open() call, it is now easy to come up with a situation where you can see a throughput stall.
Assume that the maximum number of threads in the thread pool is 100. Also, let’s say that the maximum number of connections allowed to the LOB system is 10. You receive a large number of messages from your Receive Location (for example, 500?), and BizTalk, in all its enthusiasm, queues up all messages (one message per work item – lets refer to these work items as W1) on threads from the thread pool. Therefore, you have 100 threads (suppose, since its the maximum number of allowable threads in the thread pool), all of them beginning to execute lines 1 to 6 above. Only 10 threads are able to proceed past line number 4, since they were able to open a connection to the LOB (your maximum number of connections is configured to be 10, remember?). The other 90 threads are blocked at line number 4, waiting for a connection to become available.
Of the 10 which passed 4, they proceeded to 5, which means they now enter lines 7 to 11. They all queue up work items (lets call these work items W2) on the the thread pool (courtesy line number 9), and return immediately. These 10 threads are now freed up, which enables them to go and pick more items to work on from the thread pool queue.
Now, the actual processing against the LOB (and the real usage of the connection which was successfully opened in step 4) only happens when W2 gets a chance to do its work. However, the 10 threads which were freed up, they won’t process W2. Why? Because ahead of W2, there are yet 400 more W1 work items (BizTalk had queued 500 items, remember) in the queue. Hence, the 10 threads will pick up 10 more W1 work items. These work items of course can’t progress since they are in the same state as the other 90 – there is no connection available. And a connection won’t become available until a W2 work item will relinquish it, but it can’t since it can only do its work once a thread becomes available, and that will only happen when one of the current 100 threads (which are all stuck on line 4) gets freed up, and and you have a thread pool starvation problem.
How do you work around this? A number of ways, and possibly, you need to use a combination of some of them (at least the ones in your control) to work around this:
The safest way would be to ensure that the number of messages BizTalk hands to the adapter (lines 1 to 6) is the same as the maximum number of connections allowed to the LOB. I’m not 100% sure, but I think the In-Process Messages Per CPU setting can be used for this purpose; I’ll have to go through all the documentation for the throttling settings a little more thoroughly – meanwhile, you can do the same (the link available in point number 3 above seems to be the place to start).
Hi folks,
Have I got a deal for you…. no seriously we’ve got a great night planned leading
up into TechEd. I’m really looking forward to it.
(ok let me get my dates right)
On August 27th Wednesday night (Wednesday week from now) we’re lucky
enough to get….
Angelo Laris from Decillion(seasoned member of our user group) lined up to
talk to us about A4SWIFT (which is basically what Angelo and Decillion roll
out). Decillion are formally recognised by Microsoft as being experts in this area
and are also a member of the Microsoft BizTalk Virtual Technology Specialist Program.
(I’ve also got a couple of house keeping tasks to mention..first the great stuff)
Angelo sent me through his outline:
1. Introduction to SWIFT
5min
2. What is a SWIFT Message.
5min
3. The Schema Definitions.
10-20 min
4. The HTML Form Generator (Young Youn will present or help me out here)
20min
5. The Business Rules Engine – HOW to validate a SWIFT Message.
30-60min
6. The Message Repair using Infopath and BAS.30-60min
30-60min
Looks fantastic – what you are going to see is how BizTalk can be extended, Angelo
is talking about a mature product base and it’s great to see just how others are using
and extending BizTalk while adhering to industry standards.
One of the main questions I get is “What can I use the BizTalk Rules Engine for?”..ask
no further.
What is in store for our Aug 27th Session
Where:
Microsoft, North Ryde
1 Epping Road
When:
6pm – Beer + Pizza
6.30pm – Kick off
Aug
27th –
Feel free to forward this to any of your colleagues/friends I may have missed (tell
them to register on the Sydbiz.org site to be included)
Other Business:
* TechEd 2008 – We’ve finally got RFID end-to-end at the conference (See my blog
post here – had the media interested and lots of interviews, Breeze got the green
light 40 days out from TechEd……no pressure)
Some quick stats:
1. Over 70 RFID Readers (fixed + windows mobile based) with 118 Antennas.
2. Lots of Intelligent information surfacing.
3. BizTalk Server, BizTalk RFID, BizTalk RFID V1.1, MOSS + good old Silverlight 2
b2 is pretty much running the show.
4. We’re ’printing’ (or encoding is the tech term) over 5500 tags – the hardest part
was to actually print a corresponding barcode so other systems can handle the badge.
5. I’m due for a holiday at the end of this…..(this is why I couldn’t squeeze a
meeting in last month)
So A BIG THANK YOU for those that helped out with the testing of the system (fingers
crossed on show day) at previous user group meetings.
* Call for Speakers/Other People to take Tyre Kicking Sessions
If you’ve got any aspect of BizTalk (& related) that you want to share with us….let
me know, love to hear what you’ve got to say.
* Don’t forget – we’ve launched an email forum group ’[email protected]’
Great to see a lot of you already joined – it’s a *private* group open to BizTalk
User Group members (we currently have Brisbane, Auckland and Sydney on)
Invite only – send me an email if you want to join and I’ll send you out an invite. (Thanks
to all the guys on it so far)
Love to see you there – and reply to this email to let me know for catering.
Cheers,
Mick Badran (MVP – BizTalk) | Microsoft Readiness Instructor
Collaboration and Integration Specialist
Breeze Training Pty Ltd | m: +61 404842 833
Different applications require different execution models. Some programs are quick to execute, while others may take weeks or months. Some programs have very specific steps for execution with little or no room for deviation from its set path; other programs Read More……(read more)
Hi everyone
Yes, I know that my web page is
in blue colours. Yes, I know my blog is
in blue colours. Yes, I know my new
BizTalk site is in blue colours. And yess, I know that my
sons web page is in blue colours.
Yes, I like blue colours – quit whining! 😀
—
eliasen