Selection Bias and Academic Experts

This column is so on target but also so off target: that lazy media outlets end up using quotes from the academics that they can reach instead of those that have expertise.  Yes, relying on an expert on Greece is a particularly egregious example of an ill-suited academic expert.  I know whereof I speak since I have been asked to speak on things that I know poorly, such as the likelihood of suicide bombing if the US were to invade Iraq (asked in early 2003). 

Even when I know what I am talking about and when the pre-interview focuses on questions that I can answer with some degree of certainty (even if I am wrong, at least I might be certain), the interviewer often goes off-script asking questions entirely unrelated to the pre-interview.  The same goes for how one is described.  I have tried to tell the producers that I am not a terrorism expert or an Afghan expert, but an expert on ethnic conflict and, more recently, an expert of sorts on NATO.  But the anchor/reporter will often then indicate that I am expert on things that I have never claimed to be an expert.  I make fun of myself by saying that my ten days in Afghanistan in 2007 make me an expert--it is a JOKE. 

But back to the main point, media outlets tend to go to the same well time and time again.  Sometimes this makes sense, when someone is an expert on Canadian defence procurement.  Other times, it makes less sense like the aforementioned expert on Greece. 

The problem with this particular article is that it uses this critique of the media and of academia as part of a screed about progress in Afghanistan.  That the academics are wrong, that Afghanistan is making progress.  Maybe, maybe not.  The more one learns about Afghanistan and/or about counter-insurgency, the more one realizes how hard it is.  I am sure we can find ill-informed experts saying how wonderful things are in Afghanistan as well.  So, this column really is a good example of biased selection rather than a good indictment of experts or a good argument that there is heaps of progress in Afghanistan.

Still, I will try to be a bit more cautious for at least awhile in what I say to the media since I don't want to be the poster boy for the next screed about academic experts in the media.