Kitchenham, BA, Sjoberg, D, Dyba, T, Brereton, OP, Budgen, D, Host, M and Runeson, P (2012) Trends in the quality of human-centric software engineering experiments – a quasi-experiment. IEEE Transactions on Software Engineering, 39 (7). pp. 1002-1017. ISSN 0098-5589

[thumbnail of Kitchenham-2012-TSE.pdf]
Kitchenham-2012-TSE.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (1MB) | Preview


Context: Several text books and papers published between 2000 and 2002 have attempted to introduce experimental design and statistical methods to software engineers undertaking empirical studies. Objective: This paper investigates whether there has been an increase in the quality of human-centric experimental and quasi-experimental journal papers over the time period 1993 to 2010. Method: Seventy experimental and quasi-experimental papers published in four general software engineering journals in the years 1992-2002 and 2006-2010 were each assessed for quality by three empirical software engineering researchers using two quality assessment methods (a questionnaire-based method and a subjective overall assessment). Regression analysis was used to assess the relationship between paper quality and the year of publication, publication date group (before 2003 and after 2005), source journal, average coauthor experience, citation of statistical text books and papers, and paper length. The results were validated both by removing papers for which the quality score appeared unreliable and using an alternative quality measure. Results: Paper quality was significantly associated with year, citing general statistical texts, and paper length (p <; 0.05). Paper length did not reach significance when quality was measured using an overall subjective assessment. Conclusions: The quality of experimental and quasi-experimental software engineering papers appears to have improved gradually since 1993.

Item Type: Article
Additional Information: © 2012 Institute of Electrical and Electronics Engineers (IEEE) This is the accepted author manuscript (AAM). The final published version Kitchenham, B. et al., 2013. Trends in the Quality of Human-Centric Software Engineering Experiments--A Quasi-Experiment. IEEE Transactions on Software Engineering, 39(7), pp.1002–1017 is available online via - please refer to any applicable terms of use of the publisher.
Uncontrolled Keywords: software engineering, quality evaluation, empirical studies, human-centric experiments, experimentation
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Natural Sciences > School of Computing and Mathematics
Depositing User: Symplectic
Date Deposited: 19 Dec 2016 13:48
Last Modified: 09 Mar 2020 14:31

Actions (login required)

View Item
View Item