The basic idea makes sense. That in order to evaluate whether someone has made a contribution to knowledge and is likely to continue to do so (evaluating the likely trajectory is a key part of the process), it makes sense to rely on experts who know the body of research the candidate is addressing.
After that, things go awry pretty quickly.
Here's the starting problem: that many people who write these letters do not want to be the person responsible for denying someone tenure and/or promotion. They are reluctant to write negative letters, which does mean that they may decline to write a letter to avoid doing so. Those that are willing to write negative letters or ones that are mostly positive but with critical aspects are likely to get a reputation for doing so, which then means that if they are asked to do so, it may be used by individuals and departments that are seeking to sink a candidate. Because, of course, tenure and promotion can be quite political and personal.
A second pathology is the on-going letters arms race. How many letters are necessary? In the olden days of yore, a handful seemed sufficient, like five or six. Recently, a competitive dynamic has developed whereby schools with status have, for some reason, decided that they need more letters. Like 15 or 20. Why? Do the additional letters provide more illumination or just duplication? Mostly they provide Deans and Provosts with the sense that they are running important schools that require greater efforts made by the community of scholars.
This spiral is quite problematic for several reasons:
- There may not be that many people who are experts in the specific area of the candidate
- The experts may be asked by other places to write letters so that they have to spend their summers writing heaps of letters.
- They may end up asking people who are not experts in that area, which means either weaker, less helpful letters or rejections from those who were asked.
- Because of a desire for representation (a good idea), it often leads to the few women or people of color who have tenure to get hit with lots of requests (I have a friend who regularly gets seven or more requests per year). This then presents a very unfair burden for those people.
Another status-oriented dynamic is that despite decades of bad job markets producing greater distribution of smart, sharp folks, provosts and deans insist on getting letters from peer institutions, and everybody thinks that their institution has relatively few peers, so again more requests made of fewer people. One positive consequence from moving from a more prestigious institution to a less prestigious one is that I get fewer requests than I did before. I am spewing about this mostly because my friends are hit hard by this, as I usually get between one and three requests a year.
One more status competition dynamic: that letters will often ask a letter-writer to evaluate a candidate compared to their cohort. The more prestigious places tend to provide a specific list of people to whom the candidate should be compared. This moves us pretty quickly away from evaluating whether a person is making a contribution to knowledge, but whether the person is the very best person in their field. I understand that the most prestigious places may want to only promote and tenure those who are making an outstanding contribution to knowledge, but, again, that should be about the ideas, the impact they are making on the field, and not on whether person x is better than Ken Waltz.
Oh, did I mention that except for one story I have heard, there is no compensation for this. It is part of our service to our profession.
It is obvious that we have a collective action problem as most professors do not enjoy writing heaps of letters, which get in the way of doing research, writing grants, preparing classes, and taking a swim. Wouldn't it be wonderful if the organization that seeks to represent us (that would be APSA in the US, CPSA in Canada and so on) come up with standards to ameliorate at least part of this? Like, if all poli sci departments agreed to ask for only five or six letters (yes, including Harvard), then Deans and Provosts would have to suck it up or push departments to violate the discipline's standards. This would not get rid of the other pathologies, but it would at least limit the arms race for folks trying to keep up with Harvard and Berkeley and their ilk. This would not only reduce the workload for the Sara Mitchell's of the world, but it would also make it less likely that there are overburdened people saying no, which would then make the signals less noisy, right?
I am definitely not advocating for getting rid of letters in this process. I have seen what happens in processes where there are no outside letters (my old employer used to have the letters enter the process at the level of the deans' committees), and it ain't good to not have outside expertise included in the process. But we could and should make the process less painful for all involved.
One fundamental Saideman rule is this: why do more work than is necessary?