I passed exam 70-235 today. The thing that stuck out most in my head about the exam was that it wasn’t particularly well written. Back in my Uni days I did a subject in exam analysis. I was thinking back to some of the things covered in that course as I did the BizTalk exam. In particular I remembered that all the choices should be believable and none of the choices should stick out from the others or be absurd. There were also things like checking to see if a questions was too easy (too many people got it right) or too hard. There were heaps more metrics and calculations that you perform on the responses to see if your exam is valid and that it is indeed measuring what it is supposed to measure.
Anyway I got to thinking that the BTS exam was not properly analysed as many options were absurd, questions were not clear and wording was often awkward.
What is the purpose of the beta exams if MS does not perform this type of analysis?
Whenever I ask someone involved Microsoft training and certification why they don’t do things like this properly, they say it is beacuse they don’t have the resources. I think “Why not?“. You have the resources to render entire worlds in games but you can put together a decent exam (or course for that matter – don’t get me started on their SharePoint 2003 offerings)
Anyway if you are interested in learning more about exam analysis I found this document Of course there is always plenty on wikipedia, for example http://en.wikipedia.org/wiki/Test_%28student_assessment%29 and http://en.wikipedia.org/wiki/Item_response_theory
I doubt MS take it to this extent – but they should.