Internal vs. External Validity of Research Funding

So far, most of my research funding has been from industry. Sometimes, I have to defend myself against colleagues who argue that their public funding is somehow superior to my industry funding. This is only a sentiment; they have not been able to give any particular reason for their position.

I disagree with this assessment, and for good reason. These two types of funding are not comparable and ideally you have both.

In research, there are several quality criteria, of which the so-called internal and external validity of a result are two important ones.

  • Internal validity, simplifying, is a measure of how a result is consistent within itself. Are there any contradictions within the result itself? If not so, than you may have high internal validity (which is good).
  • External validity, simplifying, is a measure of how a result is predictive or representative of the reality outside the original research data. If so, your result may have high external validity, which is also good.

Public grants have high internal validity but low external validity. In contrast, industry funding has high external validity, but low internal validity. The following figure illustrates this point:

Continue reading “Internal vs. External Validity of Research Funding”

Soundness vs. Importance in Publishing (PeerJ Computer Science Journal Announced)

Today, PeerJ announced the creation of a new open access computer science journal. After a bit of back and forth a while ago I had accepted the invitation to be on the editorial board. (My main concern was that PeerJ is a for-profit organization but co-founder Pete Binfield convinced me that this will only be used to the benefit of the authors.) A key distinguishing criterion of PeerJ when compared with other open access publishers is an all-you-can-publish rate of US$ 99 forever.

My social media stream is full of comments, positive and negative. What seems to rile people mostly are the editorial criteria, summarized succinctly as

“Rigorous yet fair review. Judge the soundness of the science, not its importance.”

Continue reading “Soundness vs. Importance in Publishing (PeerJ Computer Science Journal Announced)”

How Academics Spend Their Time? Not Me.

I just read this review of how professors spend their time while working. It struck me that a key component that I spend a substantial amount of time and energy on is missing: Fund raising. Here is a visual summary of the article courtesy of someone on reddit:

I first looked through other practices like “letter writing” and “research development” but these require no time at all so I don’t think that’s where fund raising is hiding.

I then thought that perhaps fundraising hides in meetings, making fund raising talking to industry (rather than grant proposal writing). Here is what the article says about meetings:

Continue reading “How Academics Spend Their Time? Not Me.”

German Universities to Take University Rankings Serious

Germany is the best place I know to be a professor if you value your independence. Your rights have been codified in the German Basic Law (Constitution) and no dean can tell you what to do. You are your own person.

On the downside, German professors and universities have been (for the most part) blissfully ignorant of how the rest of the world evaluates universities. Common sentiments in computer science are that “Journal publications are for wimps, real researchers publish in the leading conferences” and “University evaluations? Those are all fraudulent, focusing on crappy criteria that have no connection with reality”.

Some of these critiques are proper. For example, almost all German universities are public universites and many have a unique and positive symbiosis with industry, fueling Germany’s economic growth—where is that being accounted for in these rankings? But for the most part, Germany’s hesitance to join the international ranking game has been harmful.

In one experiment, two German universities recently decided to report their numbers to the Times Higher Education (T.H.E.) ranking with the goal of optimizing their rank. That is nothing uncommon, Northeastern University, for example, has undertaken a multi-year effort to game the US News and World report ranking, much to their benefit, apparently.

Continue reading “German Universities to Take University Rankings Serious”

Once Again Natural vs Engineering Sciences Struggeling over Definitions #FSE2014

I’m in Hong Kong, attending FSE 2014. I had signed up for the Next-Generation Mining-Software-Repositories workshop at HKUST but missed it for (undisclosed) reasons. Apparently there were two main topics of dicussion:

  • Calls by colleagues to make mining work “useful” rather than “just” interesting
  • Calls by colleagues to build tools rather than “just” generate insight

Both issues are joined at the hip and an expression of a struggle over the definition of “what is good science” in software engineering. As someone who started out as a student of physics, I have an idea of science that views “interesting insights” as useful in their own right: You don’t need to build a tool that shows your insight improves the world. On the other end is the classic notion of engineering science, where there is no (publishable) research if you don’t improve the world in some tangible way.

Continue reading “Once Again Natural vs Engineering Sciences Struggeling over Definitions #FSE2014”

Springer Verlag Adding Insult to Injury

Springer Verlag by way of its incompetence to properly edit manuscripts has been a royal pain in my butt for a long-time. In the most egregious example, one of their editors changed the title of what was a crowning paper of many years of research work. He turned “open source” into “open course”, completely altering the focus of the paper as suggested by the title. I was not given a final proof-reading chance after that change and only found out about it when I saw the paper on their website. When I complained, Springer steadfastly refused to change the title to the correct original wording and only filed an Erratum that everybody thereafter of course ignored.

Continue reading “Springer Verlag Adding Insult to Injury”

Response to Moshe @Vardi’s CACM Editorial on Open Access

In the most recent CACM editor’s letter, Moshe Vardi, the CACM’s editor-in-chief, addresses the question of open access from the perspective of the ACM [1]. The ACM is a non-profit organization for (mostly) computer scientists, and a publisher of conference proceedings and journals.

I find the editorial rather disconcerting. Vardi views “the open access movement” as being in “the IP communist camp”. There are so many things wrong this terminology. For one, I didn’t know there was one open access movement; I see many different streams of activity. Then, using 19th century terminology like communists and capitalists isn’t really going to help either; if meant as a provocation it probably achieves its goal, but to what end does this provocation help us? I’m a proponent of open access and most certainly don’t consider myself an “IP communist”. Finally, by pigeonholing well-intentioned efforts as a communist endeavor, it wholly ignores the struggle for new and innovative models of publishing research.

Continue reading “Response to Moshe @Vardi’s CACM Editorial on Open Access”

The Economics of Financing Ph.D. Students as Company Contractors

Occasionally companies approach me with the following proposal: If I’m willing to supervise one of their employees for an external Ph.D. thesis, they’ll pay into my University budget an annual lump sum, typically something like EUR 10000. I almost always reject such proposals, unless I can change some of the critical terms, because these proposals are highly problematic. To understand this, please follow along.

The company does the following math: They’ll hire someone with a recent Master’s degree, typically for a research project at the company and on a contractor basis. Then, they’ll promise the contractor that he or she can use some of their time to complete a dissertation, because they argue the project will provide enough research substance. To prove this, they’ll use the professor to confirm to the contractor that they will take them on as a Ph.D. student. A going rate for such type of contractor is (a surprisingly low) EUR 2000 per month. Times twelve months + the professor’s lump sum makes EUR 34000 per year for the company (ignoring company overhead). The official cost of a Ph.D. student at a Bavarian University is EUR 75000. Voila, the company just saved EUR 41000 a year (ignoring other University costs). However, the contractor is much worse off, because no social duties are being paid for them.

Continue reading “The Economics of Financing Ph.D. Students as Company Contractors”

Appropriate Reviewer Remuneration

As an academic, I perform a fair number of reviews. Usually, that’s part of the system, i.e. it is a give and take and fair exchange between colleagues and publishers without any monetary remuneration changing hands at all.

Then my university library complained about Elsevier’s predatory pricing and I decided to stop reviewing papers for Elsevier publications to support freedom of research and my university’s budget.

Next, I ran into the situation of wanting access to an Elsevier paper. Usually I don’t cite Elsevier papers; I just ignore them. In this case, however, I was actually curious about the paper content and wanted to use it for a talk.

Getting access to that paper took two weeks and it came too late for my talk, see this anecdotal description. However, this gave me the following idea: Why not ask for full free access to the journal in return for the reviewing service I’m providing?

Sounds like a no-brainer and I guess many folks have already asked for it. So did I, just now. Not that I’m expecting to receive a yes but it is important to make clear that predatory pricing like Elsevier’s should come to an end.

Elsevier the Unpublisher

The battle on the web over academic publishing is heating up, and Elsevier is apparently sending take-down notices to competitor If there is a publisher loathed by researchers, it is probably Elsevier. (Not so much by me, as I never published with them, but by many others whose papers they keep hostage.) I have said it before, and I am happy to repeat it here:

Papers not openly accessible on the web might as well not have been published at all.

If I cannot get to a paper through Google Scholar or other simple means, I prefer not to cite it. This includes all papers hidden behind a paywall. I consider it a moral imperative to not cite these papers for multiple reasons. For one, research funded by the public should be accessible by the public, including non-researchers. Similarly, many developing nations cannot afford the prices charged by the publishers and supporting those publishers is a way of keeping those nations in check. Thirdly, these prices are predatory, to the extent that my own university’s library asks its professors to do whatever we can to reign in the unbridled greed, specifically at Elsevier.

Continue reading “Elsevier the Unpublisher”