Using Students as a Distributed Coding Team for Validation through Intercoder Agreement [Technical Report]

Abstract: In qualitative research, results often emerge through an analysis process called coding. A common measure of validity of theories built through qualitative research is the agreement between different people coding the same materials. High intercoder agreement indicates that the findings are derived from the data as opposed to being relative results based on the original researcher’s bias. However, measuring such intercoder agreement incurs the high cost of having additional researchers perform seemingly redundant work. In this paper we present first results on a novel method of using students for validating theories. We find that intercoder agreement between a large number of students is almost as good as the intercoder agreement between two professionals working on the same materials.

Keywords: Qualitative Data Analysis, Theory Triangulation, Intercoder Agreement, Distributed Coding, Collective Coding

Reference: Andreas Kaufmann, Ann Barcomb and Dirk Riehle. “Using Students as a Distributed Coding Team for Validation through Intercoder Agreement.” Friedrich-Alexander-Universität Erlangen-Nürnberg, Dept. of Computer Science, Technical Reports, CS-2016-01, April 2016.

The paper is available as a local PDF file and also on FAU’s OPUS server.

Posted on

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Share the Joy

Share on LinkedIn

Share by email

Share on Twitter / X

Share on WhatsApp

Featured Startups

QDAcity makes qualitative research and qualitative data analysis fun and easy.
EDITIVE makes inter- and intra-company document collaboration more effective.

Featured Projects

Making free and open data easy, safe, and reliable to use
Bringing business intelligence to engineering management
Making open source in products easy, safe, and fun to use