Some Self-Imposed Reviewing Rules

 Tis that time of year: to write tenure/promotion letters.  Usually, sometime in the late spring or early summer, chairs/associate deans/etc reach out and ask academics of the right rank (tenured for tenure decisions, full profs for promotion to full) to write letters evaluating candidates for promotion.*  This is a big ask because one is expected to review someone's entire research output (outsiders are in a lousy position to evaluate teaching/service), and, no, we don't get paid to do it.

We feel obligated to do it because others wrote letters about us and because there is an understanding that saying no might be interpreted as saying that person should not be promoted.  I have written about this before, but I am writing now to provide some rules I use to get through this.  I had whined about this on facebook, and someone asked where my rules came from, which inspires this post.

But before proceeding, a few caveats:

• these are my rules--they are not the norms of the profession.  I don't know how they impact candidates since I have not been on a tenure/promotion process in some time (Carleton does this stuff by subcommittee)
• I don't get hit as hard as women and people from historically excluded groups as deans/chairs/whoever seek representation and thus pile on more work on these folks.
• I don't get hit as hard as people with bigger names or people at more prestigious places.  Moving to Carleton from McG helped me in this regard.
• I also don't get as many requests as those who have work that is quite broad.  Working on ethnic conflict then and civil-military relations now means getting requests to review people in both areas but not in an area such as "Congress" or "IR Theory."

Ok, so what are my rules/procedures?

1. I try to do only three letters a year.  Each review takes multiple days to the reading and then drafting the letter.  As a way to manage all of the stuff I have to do, containing this unpaid and unrewarded stuff to just three is necessary.  I have told my friends who get hit hard that three is enough for them too.  Given that we do this every year after tenure and we don't need that many letters ourselves (one per promotion and then maybe for a move or two), we are definitely paying far more forward.  
2. Comparison is the thief of joy.  I actually love to compare, and it is essential to my research.  But I HATE being asked to compare a candidate to other people.  In my view, academia is not a competitive bloodsport but an effort to make contributions to knowledge.  Did someone do interesting research that informed debates and moved the discussion forward?  Not did person x do more research than person y.  Some places will give a specific set of names and say: did the candidate do better than these people?   I have decided not to play this game, as I don't know the conditions at each university of each potential competitor that might facilitate or inhibit the research of the various folks involved--variations in course loads and course releases, help or lack thereof for grading, service requirements, internal funding, etc.  Some people will write letters that evaluate the work but don't recommend promotion because they don't feel they have a good understanding of the rest of the record--they stick with what they can know and say.
   
3. When I get the package, I try to figure out which five pieces to focus on.  I have finite time and I am easily distracted.  So, I am not willing to read the entire package.  And I don't think I need to. If I can't figure out if a person has made a contribution from reading either their five most well placed publications or five publications in my area of expertise, I am not going to find it in publication 7 or 10.  I have no idea how this plays with promotion committees, but I am explicit about it.  For promotion to full, I only read the stuff since they got tenure.  I don't need to re-hash the tenure decision--the question is whether they continued to contribute.
   
4. I will not dodge a negative review.  If I am asked to review someone who falls short of the tenure standards at my school (the letter requests often ask whether person x could get tenure or promotion at my place), I will still agree to write the letter.  I don't want universities to think that no letter means a negative review, and my limited capacity to influence that perception is to write negative reviews when the situations arise.  And, yes, I have done it a few times.  It was not fun, but I try to fairly assess each file--does the person make a contribution, does their trajectory look good, would I promote them at my place?  If the answers are no, no, and no, then I write the letter that way.  

What are my general standards, subject to instruction from the place requesting the letter?  Completion of an original research project--that can be a book, a series of articles, or a mix; the start of a second project.  I do check the citation record, but don't focus on it since the committee can do that.  My job is to assess the intellectual contribution of the work--have they asked relevant questions, have they developed persuasive arguments to answer those questions, have their methods been appropriate for testing their arguments, how has their work advanced the debate?  Not every aspect has to be original.  It can be a classic question with a new answer, or a new method or dataset to test existing arguments, but something has to be different than that which has been done before.  While replication is something we should do more, one's tenure should not be based on replicating previous work.

Finally, I have no idea how my tenure letters are received.  All I know is that almost everyone who I have written a positive letter for has received tenure/promotion.  The exceptions?  The places that deny most/all people--the Harvards and their ilk.  Of those who I have written a negative letter?  I think most or all of this handful have been denied.  And this is one reason why I impose limits on my work in this area--I am not sure I am making a difference (not that I want to--to be the one person standing in the way of someone's tenure).  Since much of this is pro forma one way or the other, it does not make sense that I spend my entire summer reviewing five or six tenure cases and reading every single piece of written work.  

Or at least, that is my rationalization.  I am not sure what other people use as their rules, but I wish we had more transparency on this so that we could all have reasonable expectations and so that committees and deans and provosts would understand the collective action problem they are generating.

*  I just got two requests and said no to both since I am full up.  Asking for letters in late July is not a good strategy in my humble opinion.