StreamInsight V1.1 is released (.NET Sequences / IObservable, side-by-side installation and perf improvements)

The StreamInsight dev team has just released the next version of StreamInsight (version 1.1)!!  New features in this release include:

  • .NET Sequences – no more custom adapters for common .NET data sources.  Using .NET sequences instead of adapters as event sources and sinks, which makes the integration with many existing data sources much easier and lowers the barrier of entry.
  • Side by side installation.  This enables you to safely upgrade to the new release, while having the option to keep running your existing applications against the former version of StreamInsight.
  • Performance and Stability improvements.

Read all about it on the product team announcement.

How to Define a Data Contract for Custom Serialization in WCF?

For the WCF serializer to marshal a complex type, you must define a data contract for it. To do this, start by applying the DataContractAttribute attribute to the complex type definition that you want to pass in a message. Apply the DataMemberAttribute attribute to each field (either a data member or a property) that is to be serialized. All other data fields on the class are ignored.

read more

Azure Queue Explorer

Having played a great deal with Queues in the past, the opportunity to expand the abilities of a real world solution to use the concept of Queues in the cloud as a great way of doing what Queues do best, hold onto something for someone later on to pick it up, in a managed and ordered fashion.

It seems Azure Queues do just that, and I was so happy that I made a tool that enabled me to manage my Queues:

View and Create Queues
Send and Receive Messages
Peek what messages I Have
Delete Messages from the Queue

I have made this publicly available for all to use and play with, along with the all important source code, it’s no trade secret on how to use them, it’s just nice to have something ready to go that does it for me.

Check out the link here: Azure Queue Explorer

Source code

BizTalk BAM activity Deploy problem – The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

BizTalk BAM activity Deploy problem – The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

When trying to deploy BAM activity into a new BizTalk Environment: “C:\Program Files (x86)\Microsoft BizTalk Server \Tracking>bm.exe deploy-all -DefinitionFile:BAM.SMSExpress.xls” It happened to me the following error: “Microsoft (R) Business Activity Monitoring Utility Version 3.9.469.0 Copyright (C) Microsoft Corporation. All rights reserved. Using ‘BAMPrimaryImport’ BAM Primary Import database on server ‘BTS2010LAB01′… Deploying Activity… Done. Deploying View… […]

SharePoint 2010: InfoPath Forms Services is not turned on

Here’s something that I hope to save a few hours to you – InfoPath Forms Services.

I recently ran into this dreaded error – “InfoPath
Forms Services is not turned on
” when trying to configure it from within SharePoint
Central Administration -> General Application Settings.

Forms Services is part of SharePoint Enterprise Services (usually
activated via a Farm/Web Application/Site Collection or Site feature) and it relies
upon the State Service.

So on our intranet, we’re revamping some InfoPath forms that were working in SP2007
and a new SP2010 using the database detach/attach method saw the intranet up and runningalmostexcept
for this InfoPath Forms Services.

Most posts on the web talk about simply not having the feature enabled for either
a Site collection, and/or Central Admin (and various other red herrings in my case)
http://stackoverflow.com/questions/1237956/infopath-forms-services-is-not-turned-on

http://mundeep.wordpress.com/2009/02/17/infopath-forms-services-is-not-turned-on/

Basically my source of truth was SharePoint Manager 2010 (great tool
from CODEPLEX) which allows
connections to SharePoint via the APIs as a standalone application (tip: make sure
you launch it in ’run in administrator’ mode).

From here I saw that on my install I was missing Forms Services listed
in the Farm’s Service Applications

I initially thought it was some permissions issue and that I couldn’t see the service
under that account (even though I was farm admin), so I launched and checked under
the installer account and got the same result.

My next questions were: How does Forms Services become missing? How do you
manually install/enable it?

In this case, I had a classroom SharePoint 2010 VM easily available and looking at
it through SharePoint Manager, low and behold the Forms Service service
was there!! Listed.

The machine I was having trouble with was a standard clean install, that I didn’t automatically
run the Configuration Wizard on – as I wanted to have control over
the naming of DBs. That was pretty much the difference between the two machines.

So I tried a few things:

a) installing just the InfoPath Web Admin Feature – stsadm -o installfeature
-name IPFSAdminWeb -force
and then activating it with stsadm -o activatefeature
-name IPFSAdminWeb -url
http://sp2010:10000 -force (no
luck, it just gave me the InfoPath config under the Central Admin)

b) reran the configuration wizard

c) repaired setup

d) tried to run just the InfoPath Forms Services MSI from the install source.

all to no avail.

InfoPath Forms Services – now with ’deeper’ integration with SharePoint
2010, an internal service but with no real apparent way of getting to it.

The Answer:

I got thinking and I decided to attempt a backup of the Service from my VM
and restore just the Service to the intranet Farm
.

Perform a backup of just the configuration

Then from there I did a restore (more in hope than anything) such that this process
would ’inject’ the right settings into the Farm Database.

At this point anything with InfoPath Forms Services on it was a bonus.

In any rate, here’s the backup file in ZIP format from my VM, that you can use to
restore if ever faced with a similar challenge 🙂

InfoPathFormsServices.ZIP

BizTalk 2010 Installation and Configuration – Installing and Configuring BizTalk Server (Part 8)

BizTalk 2010 Installation and Configuration – Installing and Configuring BizTalk Server (Part 8)

BizTalk 2010 Installation and Configuration, the last chapter!!! In this section you’ll install BizTalk Server, confirm that the installation succeeded, and then configure BizTalk Server. When you installed SQL Server, setup granted your account Database Administrator rights. Since these rights are also required for installing BizTalk Server, you must do one of the following: Use […]

BizTalk 2010 Installation and Configuration – Disable the Shared Memory Protocol (Part 7)

BizTalk 2010 Installation and Configuration – Disable the Shared Memory Protocol (Part 7)

Under certain stress conditions (such as clients accessing SQL Server from the same computer), the SQL Server Shared Memory protocol may lower BizTalk Server performance. You can resolve this problem by disabling the use of the Shared Memory network protocol in the SQL Server Client Network Utility. To disable Shared Memory protocol for SQL Server […]

BizTalk 2010 Installation and Configuration – Install and Configure Windows SharePoint Services (Part 6)

BizTalk 2010 Installation and Configuration – Install and Configure Windows SharePoint Services (Part 6)

Windows SharePoint Services is a collection of services for Windows Server which enable you to share information, collaborate with other users on documents, and create lists and Web part pages. For information and procedures on installing Windows SharePoint Services, see Appendix A. Note You do not need to install WSS if you do not also […]

StreamInsight: Obscure LINQ error – The field ‘Key’ of type ‘IKey`1’ either contains a nested member, or is a type that is not supported as event type field

This is a pretty obscure one, and a nice short snippet of a blog post.  I ran into this error yesterday while building out some StreamInsight applications for real time web analytics, and was very briefly stumped.  Let’s have a quick peek at how I ran into it.

Starting with a working query, I needed a different pivot/aggregation on the same data stream.  My working query was:

Code Snippet
  1. // Create the "average" quality metrics for each video from raw video quality
  2. // metrics (i.e. not quantized)
  3. var videoQualityByVideo = from e in this.videoQualityStream
  4. group e by new { e.ApplicationId, e.VideoId } into videoGroups
  5. from win in videoGroups.TumblingWindow(defaultWindowSize,
  6. HoppingWindowOutputPolicy.ClipToWindowEnd)
  7. select new
  8. {
  9. ApplicationId = videoGroups.Key.ApplicationId,
  10. VideoId = videoGroups.Key.VideoId,
  11. AverageBitrate = win.Avg(e => e.BitRate)
  12. };

Pretty basic stuff – now, I wanted to group by another field (in this case a field called EdgeServerName).  So, I copy and paste, change the grouping field name and away we go:

Code Snippet
  1. // Create the "average" quality metrics for each video from raw video quality
  2. // metrics (i.e. not quantized)
  3. var videoQualityByEdgeServer = from e in this.videoQualityStream
  4. group e by new { e.EdgeServerName } into videoGroups
  5. from win in videoGroups.TumblingWindow(defaultWindowSize,
  6. HoppingWindowOutputPolicy.ClipToWindowEnd)
  7. select new
  8. {
  9. EdgeServerName = videoGroups.Key,
  10. AverageBitrate = win.Avg(e => e.BitRate)
  11. };

Should be fine – right?  Those of you familiar with LINQ are probably already laughing at the minor syntax mistake I made, but for the rest of us here’s what happened when I tried to run the query:

Microsoft.ComplexEventProcessing.Linq.QueryGenerationException was unhandled
  Message=The field 'Key' of type 'IKey`1' either contains a nested member, or is a type that 
is not supported as event type field.  Source=Microsoft.ComplexEventProcessing.Diagnostics
  StackTrace:
Microsoft.ComplexEventProcessing.Diagnostics.Exceptions.Throw(Exception exception)       
at Microsoft.ComplexEventProcessing.Linq.GroupUnionSelectRewriter.VisitMemberAccess(MemberExpression memberExpr)
...

Ouch.  What did I do wrong?  A composite group key must have more than one field!

Right Wrong! Right
Code Snippet
  1. group e by new { e.ApplicationId, e.VideoId } into videoGroups
Code Snippet
  1. group e by new { e.ApplicationId } into videoGroups
Code Snippet
  1. group e by e.ApplicationId into videoGroups

Removing the new {} syntax for a single grouping field worked perfectly.  Another bit of LINQ learned!

Code Snippet
  1. // Create the "average" quality metrics for each video from raw video quality
  2. // metrics (i.e. not quantized)
  3. var videoQualityByEdgeServer = from e in this.videoQualityStream
  4. group e by e.EdgeServerName into videoGroups
  5. from win in videoGroups.TumblingWindow(defaultWindowSize,
  6. HoppingWindowOutputPolicy.ClipToWindowEnd)
  7. select new
  8. {
  9. EdgeServerName = videoGroups.Key,
  10. AverageBitrate = win.Avg(e => e.BitRate)
  11. };

But wait there’s more!  Through the course of the same project, I also ran into another way to throw this error, in a slightly different fashion.

Code Snippet
  1. var videoQualityByEdgeServer = from e in this.videoQualityStream
  2. group e by new { e.ApplicationId, e.EdgeServerName } into edgeGroups
  3. from win in edgeGroups.TumblingWindow(videoQualityEventWindowSize, windowAlignment,
  4. HoppingWindowOutputPolicy.ClipToWindowEnd)
  5. select new
  6. {
  7. ApplicationId = edgeGroups.Key,
  8. EdgeServerName = edgeGroups.Key.EdgeServerName,
  9.  
  10. // Buffering Milliseconds -> Seconds
  11. // Divide by sampling frequency seconds
  12. // Multiply by 100 (1 / 1000 * 100 = 1 / 10)
  13. AvgBitrate = win.Avg(e => e.BitRate)
  14. };

If you have a composite key, and don’t use the individual properties of the grouping key, the error is thrown.  Easy in retrospect, right?