The JDownloader Immune System for Continuous Deployment (HICSS 53)

Abstract: Continuous deployment can reduce the time from a source code change to a newly deployed application significantly. Increased innovation speed can make all the difference in a competitive market situation. However, deploying at high frequency requires high speeds of discovering bugs in the deployed software. Using the JDownloader file download manager as our example, we present a fitness model to evaluate a continuously deployed software during operation for expected behavior, present the design and implementation of a monitoring component, and evaluate the model and its implementation using data from JDownloader’s multi-million member strong user base. Our evaluation finds that there had been thousands of undetected bugs, and that newly created bugs can be detected and reported 16 times faster than before.

Keywords: Continuous deployment, continuous delivery, immune system

Reference: Rechenmacher, T., Riehle, D., & Weber, M. (2020). The JDownloader Immune System for Continuous Deployment. In Proceedings of the 53rd Hawaii International Conference on System Sciences (HICSS 2020), to appear.

Download: The paper is available a PDF file.

Agile Methods and the Magic Triangle

In software engineering, the magic triangle is a well-known concept to illustrate the relationship between scope, time, and cost of a software development project. Of the three (scope, time, cost), pick two, and the third will magically follow. (It is determined by the other two.) Scope means features (or delivered functionality), time means duration or deadline, and cost typically means number of developers or, more abstractly, available labor.

Continue reading “Agile Methods and the Magic Triangle”

Scrum’s Product Vision vs. Project Mission

As noted previously, Scrum uses the term product to mean artifact. This is fine, as long as the user of Scrum is a software vendor, developing a product for a market. It is confusing, however, if the user is a consulting firm, performing a custom project for a client. If you are a consulting firm, you are not delivering a product, and every time you hear product, you need to think artifact.

The confusion is worst when we talk about a product vision. As always, there are many competing definitions and confused ones at that, but we can safely assume that a product vision is at least a particular type of vision. About a vision, by definition, we know that it is time-less. It describes an abstract future state that we want to achieve, but never actually can reach. As such, a vision serves as a guiding north star for the decisions we make about on-going work. IBM’s vision is (shortened) “client success”, SAP’s is “improved economy”, etc. Any number of management books expound on what a vision is so you can read more there.

Continue reading “Scrum’s Product Vision vs. Project Mission”

How Project vs. Product Confuses Agile Methods Terminology

In a previous blog post I noted how the terms project and product are being confused in open source. However, it is agile methods, specifically Scrum, where it gets really bad. To recap: A project is a human undertaking to create an artifact. A project, by definition, has a start date and an end date. A product, in contrast, is an artifact (not an undertaking) that is born but typically has no planned end date. Most vendors, selling products to a market, hope they can do this forever.

Continue reading “How Project vs. Product Confuses Agile Methods Terminology”

Best of “As a Developer, I Want…” Student User Stories, Raw and Unfiltered

Over on Twitter, the ever amazing and amusing time sink, @AdamPridmore, @KevlinHenney, and @JensColdewey are discussing the merit of software developers as stakeholders in user stories.

Despair not! I’ve got the goods. From my Scrum student projects (I run those day-in, day-out), here is the classic one:

As a developer, I want to learn Java to be able to program and earn a top grade.

I looked through the last few feature archives (the place where stories go to rest) from my student projects. Here are some fine exemplars of developer user stories. Have fun!

Continue reading “Best of “As a Developer, I Want…” Student User Stories, Raw and Unfiltered”

Clarification of “Why I Still Teach Scrum”

Teaching Scrum at University is challenging. Students are typically at the beginning of their career and don’t understand the challenges of communication and coordination in software engineering well. In a prior post on Why I Still Teach Scrum I had made a cryptic remark to that end and through various channels was asked to clarify the remark.

Scrum is a framework that needs tailoring for the specific situation at hand. The specific challenges of teaching Scrum at a University are, in no particular order, and most certainly incomplete:

  • Comparatively little practical experience of students
  • Widely differing capabilities as the gap between students can be large
  • Transient teams as Scrum projects typically last only a semester
  • Zero-Sum-Thinking by some students as to teamwork and grade

Continue reading “Clarification of “Why I Still Teach Scrum””

Agile Feature Teams vs. Inner Source

Agile methods reacquainted developers with the idea of working from business value rather than focusing on technical concerns only. Agile methods are therefore often equated with feature-driven development, in which work is driven by features prioritized by business value irrespective of technical consequences. This thinking can create code silos and wreak havoc on software architecture and component quality. Developer complaints are legion, in particular for never getting the time to fix things or do them right in the first place.

Continue reading “Agile Feature Teams vs. Inner Source”

Why I Still Teach Scrum

Scrum is an agile method (framework) that when instantiated can be rather ornate. Most developers, when I talk to them, tell me that when given a choice they would not be doing Scrum. While Scrum may have felt much lighter than the competition back in the nineties, today it weighs in as rather heavy.

Given this, I wanted to reflect on why I still teach Scrum (and have a blog post to point any of my students to).

Continue reading “Why I Still Teach Scrum”

Teaching Materials for Agile Methods Course

Update 2018-08-01: The old materials are not available any longer; please watch this space for the new release!

I finally put my teaching materials for my agile methods course on this website. The slides are available in “source” form, i.e. Open/LibreOffice format, as well as PDFs. I also added supplementary materials like the videos I use for illustration purposes. The slides are made available using the Creative Commons BY-SA license and are based on a course I’ve been giving several times now. It is far from being perfect but obviously good enough for a real course. Feel free to use or copy from the slides for your own courses!

My goal is to keep improving the slides. I expect there to be a new version every year or maybe every semester. For me, this is an experiment. I honestly don’t know how to collaborate around a format like ODP and ODT. It sure doesn’t feel like source code. So, my best suggestion is that if you find this useful and would like to see it improve in a direction that suits you, please let me know of your suggestions. I might then incorporate the suggested changes into the slide set. In general, my philosophy is that the content will grow, but ideally in a consistent fashion.

Agile Methods Course at Tsinghua University

Update 2018-08-01: The old materials are not available any longer; please watch this space for the new release!

Update 2012-03-28: I made the course slides available to the public.

I just finished teaching a one-week course on agile methods at Tsinghua University, the top (mainland) Chinese engineering school and one of the two leading Chinese universities. My host told me that I was the first non-Chinese-speaking lecturer to have held such a short course, not only in Computer Science but at Tsinghua as a whole. (I’m sure there have been plenty of prior foreign lecturers, but apparently I was the first one not to teach for a whole semester, but only for this condensed one-week half-day type of course). Yay! Adventure and breaking new grounds is still possible on this planet.

Moreover, with my research partner Prof. Bai, I’ll be leading a joint distributed agile software development project, involving student teams from both Tsinghua University (THU) and Friedrich-Alexander University (FAU). The goal of the project is to learn about what makes or breaks distributed agile development. We’ll start with simple hypotheses but hope to grow this into something larger. We already have student teams, but are looking for more. If you are a software engineering student at either THU or FAU, please come and talk to us!