Madeyski, L and Kitchenham, BA (2017) Would wider adoption of reproducible research be beneficial for empirical software engineering research? Journal of Intelligent and Fuzzy Systems, 32 (2). pp. 1509-1521. ISSN 1064-1246

[thumbnail of kitchenham_2017.pdf]
kitchenham_2017.pdf - Published Version
Available under License Creative Commons Attribution.

Download (1MB) | Preview


Researchers have identified problems with the validity of software engineering research findings. In particular, it is often impossible to reproduce data analyses, due to lack of raw data, or sufficient summary statistics, or undefined analysis procedures. The aim of this paper is to raise awareness of the problems caused by unreproducible research in software engineering and to discuss the concept of reproducible research (RR) as a mechanism to address these problems. RR is the idea that the outcome of research is both a paper and its computational environment. We report some recent studies that have cast doubts on the reliability of research outcomes in software engineering. Then we discuss the use of RR as a means of addressing these problems. We discuss the use of RR in software engineering research and present the methodology we have used to adopt RR principles. We report a small working example of how to create reproducible research. We summarise advantages of and problems with adopting RR methods. We conclude that RR supports good scientific practice and would help to address some of the problems found in empirical software engineering research.

Item Type: Article
Additional Information: This is the final published version of the article (version of record). It first appeared online via IOS Press t - please refer to any applicable terms of use of the publisher.
Uncontrolled Keywords: reproducible research, empirical software engineering, scientific practice
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Natural Sciences > School of Computing and Mathematics
Depositing User: Symplectic
Date Deposited: 23 Jan 2017 11:00
Last Modified: 02 Apr 2019 14:49

Actions (login required)

View Item
View Item