Tenure Letters and Cohort Comparisons: This Way Lies Confusion

Tenure and promotion letters are one of the services academics do when once they get past tenure themselves.  I have blogged in the past about whether or not to write these letters, so today's post is about a frequent challenge when writing such letters: some (many?) provosts/deans/whoevers ask the letter writers to compare the candidate to the candidate's cohort--other people in the same area of research who have been at it for a similar time.*

*This post only addresses research since outside letters can only speak to research and because my only experience has been in research universities.

This request poses both practical and normative challenges.  The practical challenge is this: how does one know who the comparative cohort is?  As far as I know, there is no handy search engine that will pop out names of people in a subfield or research area sorted by year of PhD completion.  I don't have an encyclopedic memory for who finished in what year, nor, because I am far behind in my journal reading, really know who is doing what.  Reading all of the materials is extra work enough without systematically going through "peer institutions"** and identifying folks in the relevant subfield who are at the same stage of their career. I posted on facebook, essentially asking my IR friends for names of folks who would be in this person's cohort.  Instead of giving me names (ok, one or two people did), this led to a long and interesting discussion of the entire exercise of comparing.

**  One of the basic problems in all of this is that every Dean/Provost has an inflated sense of what their institution is, so the list of peer institutions is quite small--the Ivies, the top public schools and a few others.  It does help me, however, that I moved from a school seen as peer (McGill) to one that is not (Carleton), so I get fewer requests now than I did at the old place.  Woot.

The folks arguing for comparing to a cohort argued this was one of the most valuable pieces of information in the letter since everyone mostly writes super positive letters lest their few criticism arm those who are opposed to a candidate for whatever reason (not infrequently illegitimate ones like sexism, racism, animus, retaliation, etc).  More importantly, some folks argued that to evaluate a candidate, they should be compared to their peers.***  This is what many letter requesters want, and some even name specific scholars (usually the most well known/cited/productive).   Even if focused on a person's contributions sans comparison, competition ultimately enters as one evaluates the quality of the presses in which the candidate publishes, the selectivity of the journals in which their work appears, citation counts and h-indexes are essentially comparative and so on.

***One friend argued with me that competitiveness is productive, that folks who are competitive will be motivated to continue to publish after tenure, and that those who are not motivated by comparing themselves to others are likely to become deadwood.  I think curiosity and professionalism bred into us is sufficient, but I am sufficiently ego-driven that I see something to that argument.

But this raises a question of what is the point of being a scholar, of being promoted and tenured?  To be better than others?  Or to be productive, to make a significant contribution?  What difference does it make if candidate x is not as productive as the most productive people in the discipline?  Not everyone can be above average.  Perhaps the idea is only to tenure/promote people who are above the people who are at the average level of productivity?  How I write the letter depends on how I see the profession, and while there is a heap of competition in it--to get into grad school, to get grants/fellowships, to get into the more selective journals and presses, to get jobs--I think the larger enterprise is not competitive. It is about making contributions to knowledge, building on the work of others (past and present).  That co-authoring, for instance, and other forms of collaboration should not be penalized (I wrote the linked post in the aftermath of my co-authored work being dismissed by my senior colleagues because .... motivated bias, so that post might be a bit strident).  Moreoever, as one friend argued, relative comparisons may be unfair when there is a heap of bias--in who gets cited, who gets published in the top presses, etc. 

The tenure/promotion letter, in my view, is about addressing whether this person has made a contribution and is likely to continue to make a contribution. To me, these are absolute questions, not relative ones.  Which is why most of the letter is about what the person has researched and written and what their stuff contributes rather than the bean counts and comparisons with cohorts.  When asked to compare, I try to do so because, like saying no to the request, not following the instructions can be seen as criticism. But I don't like it, and I have a hard time because I do not have a good grasp of who is in the cohort.  So, what else do I do?  I whine here about it.