Rigor vs. Relevance, or: What is the Size of a Dissertation?

While listening to a colleague’s talk the other day, I got an idea for a Ph.D. thesis (grant proposal). I wrote up a short summary and sent it to him. He thought it was fine but commented that it might be a bit “thin”. This made me wonder: How do we determine sufficient size of a dissertation, to stay with the metaphor of thin, so that we can conclude some research work is worth a Ph.D. title? Most university regulations require “significant” (read: non-trivial) scientific progress and then leave it to the advisor and the reading committee to determine whether a submitted dissertation fits the bill.

My first reaction was to take a visual approach: Size = Breadth x Depth. If a thesis has sufficient breadth (of topic) that it treats with enough depth, then it covers sufficient research area and should be accepted. Be very broad but not deep (i.e. shallow, lack of rigor in validation of your claims) and your research surface area goes down. Be very deep but on a trivial subject (i.e. claims are overly specific and “thin”) and your surface area also goes down. Thus, you’d want a decent balance of breadth and depth to cover a sufficient area.

This line of thinking reminded me of Nailing your Thesis (NYT), the research class I teach. There, I use the rigor vs. relevance debate to explain to students how to think about a paper’s contributions (and to get it past reviewers). In this debate, research is being assessed along two dimensions, one being the rigor (of research) and one being its relevance. Pretty close to what I meant by depth (rigor) and breadth (relevance) above. In NYT I use the following graph to show how you need to balance rigor vs relevance to get your work accepted:

Rigor means the rigorousness of your work: Do you validate your claims? To what extent? Are you just providing usage examples that make your claims plausible or did you perform a thorough experiment where you were able to control for most variables and assess how the issue of interest really behaved? Relevance means how important your result is: Did you address some niche topic nobody ever heard of or did you solve a truly interesting and important problem?

Naturally, scientists want to see both but tend to err on the side of rigor. It is better to have a smaller but well-defined result rather than a broader more important result that isn’t trustworthy. The fringes are problematic: Neither “proving everything about nothing” (100% rigor, 0% relevance) nor “proving nothing about everything” (0% rigor, 100% relevance) will get your paper accepted, not to mention a Ph.D. title. As the graph above suggests there is an efficient frontier that you should reach in a well-balanced combination of rigor and relevance. And that’s your dissertation’s minimal size.

Posted on

Comments

  1. Neil Avatar
    Neil

    Here a PHD thesis in SE is three papers (+/- 1) in the top venues in the field: ICSE, ASE, FSE, RE, etc. This leaves it up to others to assess breadth and depth.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Share the Joy

Share on LinkedIn

Share by email

Share on Twitter / X

Share on WhatsApp

Research Projects

Making free and open data easy, safe, and reliable to use
Bringing business intelligence to engineering management
Making open source in products easy, safe, and fun to use