The Hardest Part of Starting a Research Project

 I originally asked what the hardest part of doing social science is and folks ended up talking about policy relevance and getting published and getting grants.  So, to be clear, my focus was and is for this post--how to get started and what is the hardest mental/brainpower part. As a prof running a PhD dissertation proposal workshop, as a supervisor of nearly 20 PhD students over my career, and as a researcher, I thought the answer was obvious.  But nope, it ain't:

 What is the hardest part of doing social science?

— Steve Saideman (@smsaideman) February 16, 2021

Turns out all three parts--the question, the answer, how to figure out if the answer is right--are all seen as being the hardest part by roughly an equal percentage of people who answered the survey.  Again, others suggested other parts of the process, but every research project involves at least these three pieces.  And again, I was really trying to ask about what is the hardest mental exercise--figuring out what to study, how to think about it, or how to design the research?

I asked this survey out of frustration--that many of my current students are so focused on the methods that they skip over the thing that logically comes before methods: what is your proposed answer to the question?  What is your theory of x?  What do you think is the key causal mechanisms at work, what are the key conditions associated with the outcome, and why?  What is the logic of all of this?  What is the logical glue that holds together the various things that you think matters most?* 

I do think that coming up with research questions is hard, especially early in one's career. Dr. Wahedi's points here are really important.

But once you've been around a while, started developing your own work, you have a stack of questions that fit into your broader theoretical framework and are limited by what you can get an identification strategy for, or what you can pull off within budget/data constraints,

— Dr. Laila Wahedi (@lwahedi) February 17, 2021

That it is hard to figure out what to ask that hasn't been asked before.  Or, more likely, what has been asked but not answered satisfactorily that one can answer?  Of course, that already begins to bleed into the theory--do I have an answer that hasn't been posed or posed well before? As Dr. W suggests, this probably gets easier over time.  I have found it far easier now than when I started to figure out what questions are interesting and have not been answered satisfactorily.  I now have too many questions, not enough time to figure out the answers and then do the research.

The methods should be driven by the theory.  It can be hard to figure out how to test a theory, but, it is, of course, impossible without having some observable implications derived from a theory to try to, um, observe.  Once one has hypotheses, figuring out to test them can be much work.  Is there a dataset?  Can one construct a dataset?  What kinds of case studies make sense?  Most likely?  Least likely?  A sample representing the different possibilities (one case for each cell of a 2 by 2)?  Outliers?  There are plenty of books and articles that explain not just the methods but how to select which ones.  Why choose negative binomial distribution rather than poisson?  

There are far fewer guides for how to think theoretically.  While there are plenty of courses aimed at improving one's ability to think theoretically, none are so explicit about step 2 as methods courses are regarding step 3 (unless one gets a purely theoretical version of a methods course, like, alas, I got long ago).

Perhaps one reason why my Phd students in the past found this harder than other students is that I am not a disciple nor insist that my students be disciples.  I am theoretically eclectic as my supervisor was before me, and so I can't tell my students to think in a particular way.  If I did, then the challenge they would have faced is: what is a good question to apply our favorite theory to and what are the methods to test it.  For the proposal class, this is doubly true since the students range across International Affairs, from largely economic studies on trade or investment to mixed questions to mostly poli sci type questions.  So, there definitely is no magical theory that all students should be adopting.  Indeed, many of them apply pieces of theories that I am most unfamiliar with.   Which then makes it harder for me to help them through this troublesome stage.  

However, the survey suggests that I was wrong about which part of the research thinking process is the hardest.  Ironically, I have found an interesting question--why do people see the different stages differently?  I could develop a theory, but that would require work.  So, we don't really know what is the hardest part.  Tom Petty said it was the waiting, but Blondie says otherwise.

* My view of all of this is, of course, biased as I am a positivist.  One of my struggles with post-positivism is understanding how their pieces fit together.