Many computer science degree programs do a lousy job at teaching science. A high school student, entering university, often has a good idea what science is about, based on their physics and chemistry classes. At least, it involves controlled experiments. At university, this is rarely picked up, and computer science students are given the idea that programming something novel constitutes science. With that idea, they are often bewildered when I teach them rigorous research methods, in particular if those originated in the social sciences (like qualitative interviews or hypothesis-testing surveys).
Ever since I have looked for simple ways of explaining the process of traditional (behavioral / descriptive) science to students. Epistemologically, it should be easy! Computer science is deeply rooted in the world-view of an objective truth and a reality independent of the researcher. Positivism may be too simple for some disciplines, but it forms the foundation of research in computer science.
With that, here is another attempt at explaining traditional science.
At the most basic level, we separate theory building from theory validation (see picture). A theory, at its simplest, is a description or model that can explain current events or predict future ones. It is abstract knowledge used to generate specific insights. It is key to this definition that a theory must be able to explain or predict something, otherwise there would be no way of testing the theory. During theory building, we construct the theory in an iterative ideally theory-expanding cycle of initial theory creation, evaluation, and revision.
Key to science is the use of proper research methods. If we don’t use research methods (that should have been validated themselves as being able to reliably generate the results they promise), we are just phantasizing stuff. A research method for theory creation and revision, for example, is the qualitative survey. A more complex research method for theory building, involving all of theory creation, evaluation, and revision is case study research.
Within theory building, the purpose of theory evaluation is to identify weaknesses and gather information for theory revision and expansion.
I view theory validation as separate from theory evaluation. During theory validation, a researcher tests the theory being built. Such tests take place by way of hypotheses that are generated from the theory and then tested. If the hypothesis test fails, we know that something needs fixing about the theory.
This separation of theory evaluation, which is part of the theory building cycle, and theory validation, which tries to ascertain truth of the theory, is important, because it is rooted in the research methods applied. Theory evaluation typically uses qualitative research methods that have little to no claims to broad generalizability, while theory validation uses hypothesis-testing research methods that within a defined scope very well claim, if done right, to be representative of some population. Thus, the purpose of theory evaluation is to power the next cycle of theory revision and expansion, while the purpose of theory validation is to ascertain truth.
It is not here yet, but this is part of my course on performing research, called “nailing your thesis”, to be made available on this site next to my other open courses.
Leave a Reply