How to Slice Your Research Work for Publication

I often discuss with my Ph.D. students how to structure their work and publish the results. There are many pitfalls. It gets more difficult, if we bring in other professors, who may have a different opinion on how to structure the work. Over time, I have found there are really only two main dimensions, though:

  • Separate work into theory building and theory validation, and publish one or more papers on each topic
  • Merge work on theory building and validation for an important aspect (hypothesis) into one and publish that

Continue reading “How to Slice Your Research Work for Publication”

Third Time’s the Charm: Ten Years of Inner Source Case Studies

Actually, I just notice it is the fourth time within the last two months, but tomorrow is the first time I’ll present our research on inner source in a public venue. If you are interested in ten years of case studies on how to use open source best practices within companies (called inner source), come see me at the Bitkom working group meeting on open source at Design Offices, Unter den Linden 26-30, 10117 Berlin.

How the Lack of Theory Building in Software Engineering Research is Hurting Us

Traditional science has a clear idea of how research is to progress, rationally speaking: First you build a theory, for example, by observation, expert interviews, and the like, and then you generate hypotheses to test the theory. Over time, some theories will stand the test of time and will be considered valid.

Sadly, most software engineering research today, even the one published in top journals and conferences, often skips the theory building process and jumps straight to hypothesis testing. Vicky the Viking, from the accordingly named TV series of way back comes to my mind: Out of the blue, some genius idea pops into the researcher’s mind. This idea forms the hypothesis to be tested in research.

Continue reading “How the Lack of Theory Building in Software Engineering Research is Hurting Us”

The QDAcity-RE Method for Structural Domain Modeling Using Qualitative Data Analysis

Abstract: The creation of domain models from qualitative input relies heavily on experience. An uncodified ad-hoc modeling process is still common and leads to poor documentation of the analysis. In this article we present a new method for domain analysis based on qualitative data analysis. The method helps identify inconsistencies, ensures a high degree of completeness, and inherently provides traceability from analysis results back to stakeholder input. These traces do not have to be documented after the fact, but rather emerge naturally as part of the analysis process. We evaluate our approach using four exploratory studies.

Keywords: Domain modeling, Domain model, Requirements engineering, Requirements elicitation, Qualitative data analysis

Reference: Kaufmann, A., & Riehle, D. (2017). The QDAcity-RE method for structural domain modeling using qualitative data analysis. Requirements Engineering, 1-18.

The paper is available as a PDF file.

Why are There Only Two Research Groups Working on Inner Source?

I got asked the other day why there are only two research groups working on inner source world-wide. Inner source is the use of open source best practices within companies, and it is a hot topic with many companies who want to go beyond agile. There was varied research around the world in the past 15 years, but only two groups really have been consistently working on this: Brian’s group at LERO and my research group at FAU.

Continue reading “Why are There Only Two Research Groups Working on Inner Source?”

Why “Soft” Research is “Hard”

Some of my colleagues like to talk about how research that involves programming is “hard”, while research that involves human subjects is “soft”. Similarly, some colleagues like to call exploratory (qualitative) research “soft” and confirmatory (quantitative) research “hard”. Soft and hard are often used as synonyms for easy and difficult, and this is plain wrong.

Pretty much any research worth its salt is difficult in some way, and working with human subjects makes it even more difficult. I find research methods like qualitative surveys, involving interview analyses, for example, much harder than the statistical analysis of some data or some algorithm design. The reason is the lack of immediate feedback.

While you can (and should) put quality assurance measures in place for your interview analyses, ranging from basic member checking to complex forms of triangulation, it might take a long time until you learn whether what you did was any good. So you have to focus hard in your analysis without knowing whether you are on the right track. It doesn’t get more difficult than this.

Understanding Industry Requirements for FLOSS Governance Tools

Abstract: Almost all software products today incorporate free/libre, and open source software (FLOSS) components. Companies must govern their FLOSS use to avoid potential risks to their intellectual property resulting from the use of FLOSS components. A particular challenge is license compliance. To manage the complexity of license compliance, companies should use tools and well-defined processes to perform these tasks time and cost efficiently. This paper investigates and presents common industry requirements for FLOSS governance tools, followed by an evaluation of the suggested requirements by matching them with the features of existing tools. We chose 10 industry-leading companies through polar theoretical sampling and interviewed their FLOSS governance experts to derive a theory of industry needs and requirements for tooling. We then analyzed the features of a governance tools sample and used this analysis to evaluate two categories of our theory: FLOSS license scanning and FLOSS in product bills of materials. The result is a list of FLOSS governance requirements based on our qualitative study of the industry, evaluated using the existing governance tool features. For higher practical relevance, we cast our theory as a requirements specification for FLOSS governance tools.

Keywords: Open Source Software, FLOSS, FOSS, Open Source Governance, FLOSS governance tools, company requirements for FLOSS tools

Reference: Nikolay Harutyunyan, Andreas Bauer, and Dirk Riehle. 2018. Understanding Industry Requirements for FLOSS Governance Tools. In OSS ’18: 14th International Conference on Open Source Systems, June 8-10, 2018, Athens, Greece. Springer, IFIP Advances in Information and Communication Technology, 12 pages.

A preprint of the paper is available here as a PDF file.

The Patch-flow Method for Measuring Inner Source Collaboration

Abstract: Inner source (IS) is the use of open source software development (SD) practices and the establishment of an open source-like culture within an organization. IS enables and requires developers to collaborate more than traditional SD methods such as plan-driven or agile development. To better understand IS, researchers and practitioners need to measure IS collaboration. However, there is no method yet for doing so. In this paper, we present a method for measuring IS collaboration by measuring the patch-flow within an organization. Patch-flow is the flow of code contributions across organizational boundaries such as project, organizational unit, or profit center boundaries. We evaluate our patch-flow measurement method using case study research with a software developing multi-industry company. By applying the method in the case organization, we evaluate its relevance and viability and discuss its usefulness. We found that about half (47.9%) of all code contributions constitute patch-flow between organizational units, almost all (42.2%) being between organizational units working on different products. Such significant patch-flow indicates high relevance of the patch-flow phenomenon and hence the method presented in this paper. Our patch-flow measurement method is the first of its kind to measure and quantify IS collaboration. It can serve as a base for further quantitative analyses of IS collaboration.

Keywords: Inner source, internal open source, inner source measurement, patch-flow, open source, open collaboration, software development collaboration measurement, inner source metrics

Reference: Maximilian Capraro, Michael Dorner, and Dirk Riehle. 2018. The Patch-Flow Method for Measuring Inner Source Collaboration. In MSR ’18: 15th International Conference on Mining Software Repositories , May 28–29, 2018, Gothenburg, Sweden. ACM, New York, NY, USA, 11 pages.

A preprint of the paper is available here as a PDF file.

Why Companies Don’t Always Free-ride on Open Source Projects

I presented on open source foundations earlier this week to economist friends at TU Munich. I naturally got the question about freeriding: Why does anyone contribute to open source projects, if they could do something else with their time? The cinch: This time we are talking about companies, not invididual people, so the arguments about altruism and signaling don’t apply. So, why do companies contribute and don’t just freeride? I don’t think this question has been answered well yet in economics, and I’m not sure established theory has a ready answer.

To make it short: I believe the most direct reason why companies contribute to open source projects is to lower their cost of consumption of that very project. Specifically, contributing to a project builds competence in that project, and employing committers builds additional foresight and influence. General compentence makes the company use the software more effectively, avoiding costly bugs and rework. Foresight and influence helps the company avoid misalignment of their products with the evolving open source software they depend on. Such misalignment can also lead to costly rework and missed market opportunities.

I’m not aware of any RoI model that helps an engineering manager determine how much to contribute to achieve how much lower consumption costs and risks. Because of the step function from contributor to committer status for the involved employees, the investment return is not a linear function, that much we can say. The rest remains imperfect science for now.

Continue reading “Why Companies Don’t Always Free-ride on Open Source Projects”